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Secure Transmission of Sources 
over Noisy Channels 
with Side Information at the Receivers 

Joffrey Villard, Pablo Piantanida, and Shlomo Shamai (Shitz) 

Abstract 

This paper investigates the problem of source-channel coding for secure transmission with arbitrarily 
correlated side informations at both receivers. This scenario consists of an encoder (referred to as Alice) 
that wishes to compress a source and send it through a noisy channel to a legitimate receiver (referred to 
as Bob). In this context, Alice must simultaneously satisfy the desired requirements on the distortion level 
at Bob, and the equivocation rate at the eavesdropper (referred to as Eve). This setting can be seen as a 
generalization of the problems of secure source coding with (uncoded) side information at the decoders, 
and the wiretap channel. A general outer bound on the rate-distortion-equivocation region, as well as 
an inner bound based on a pure digital scheme, is derived for arbitrary channels and side informations. 
In some special cases of interest, it is proved that this digital scheme is optimal and that separation 
holds. However, it is also shown through a simple counterexample with a binary source that a pure 
analog scheme can outperform the digital one while being optimal. According to these observations and 
assuming matched bandwidth, a novel hybrid digital/analog scheme that aims to gather the advantages 
of both digital and analog ones is then presented. In the quadratic Gaussian setup when side information 
is only present at the eavesdropper, this strategy is proved to be optimal. Furthermore, it outperforms 
both digital and analog schemes, and cannot be achieved via time-sharing. By means of an appropriate 
coding, the presence of any statistical difference among the side informations, the channel noises, and 
the distortion at Bob can be fully exploited in terms of secrecy. 
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I. Introduction 

Consider a system composed of three nodes (or sensors) where each one is measuring an 
analog source (or random field) as a function of time. One of them (referred to as Alice) wishes 
to transmit a compressed version of its observation to a second node (referred to as Bob) through 
a noisy (or wireless) channel. In addition, Bob can use his own observation as side information 
to decode the received message and refine his estimate of Alice's source. The third node (referred 
to as Eve) is an eavesdropper i.e., a node that can listen to the messages sent by Alice through 
another noisy channel. Considering that Eve is not to be trusted, Alice wishes to leak the smallest 
amount of information about her source. 

Among some major information-theoretic issues, the above scenario involves the notion of 
secrecy (and its application to source and channel coding), source coding with side information, 
as well as joint source-channel coding for transmission of sources over noisy channels. The 
information-theoretic notion of secrecy, introduced by Shannon [0Q, was first used for secure 
communication over noisy channels by Wyner 0, who studied the so-called wiretap channel. In 
particular, allowing the encoder to introduce some independent random noise in the transmitted 
signal, Wyner showed that it is possible to send information at a positive rate with perfect secrecy 
as long as the channel of the eavesdropper is a degraded version of the legitimate user's one. 
Csiszar and Korner [3] extended this result to the setting of general broadcast channels with 
any arbitrary level of security, as measured by the equivocation rate -the remaining uncertainty 
about the message- at the eavesdropper. Extensive research has since been done, yielding several 
extensions of the original wiretap channel l|4ll-lf8l. 

On the other hand, source coding with side information has been studied by Slepian and 
Wolf [9'], and Wyner and Ziv iflOll . Security constraints with respect to an additional eavesdropper 
that must be kept as ignorant as possible of the transmitted source were recently introduced in 
these source coding problems [fTTTl — [TT4TI . The optimal coding scheme has been characterized in 
the general case lfT3l . Ifl4l . It uses standard coding techniques (superposition coding, random 
binning, etc.) as well as a new evaluation of the equivocation rate at the eavesdropper. As a 
matter of fact, if the side informations at the decoders are degraded, then Wyner-Ziv coding IfTOl 
is optimal, while it is proved to be insufficient in the general case. 

As in the papers cited above, most of the existent work separately considers channel or source 
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coding for secure transmission or compression. However, unlike point-to-point communication 
problems lfT5l . lfT6l . there is no general result of separation for multiterminal settings under 
security constraints. Recent work IfTTl considered such a setting of source-channel coding for 
secure transmission by assuming that Eve has a degraded channel with degraded side information 
with respect to Bob, and shows that separation holds. Along the same line of work, state 
amplification subject to masking constraints, where Alice wishes to effectively convey -amplify- 
the channel state sequence to Bob while masking it from Eve, has been investigated in lfT8l . This 
may indicate that digital schemes are well-suited for these multiterminal settings with security 
constraints. On the other hand, it is well-known that joint source-channel coding/decoding is 
a must for broadcast channels without secrecy constraints lfl9l . Il20l . and hybrid digital/analog 
schemes have been proved useful for point-to-point problems e.g., to handle SNR mismatch 
(while they can perform as good as digital or analog ones at the true SNR) [|2TT|. |[22|. as well 
as for some multiterminal settings Il23l - ll25l . By taking advantage of both analog and digital 
strategies, they may help to solve the considered problem for secure transmission in the more 
general case without any degradedness condition. 

In this paper, we consider the setup of joint source-channel coding for secure transmission of 
a source over a noisy channel with an eavesdropper, and in the presence of side information at 
the receiving terminals, as depicted in Fig. Q] This setting can be seen as the unification of the 
problems of secure source coding with side information at the decoders lfT3l . fljj, and the wiretap 
channel [0, 0. The main goal is to understand how Alice can take simultaneous advantage 
of the statistical differences among the side informations and the channel noises to reveal the 
minimum amount of information to Eve, and satisfy the required distortion level at Bob. It should 
be emphasized that the central difficulty of this problem lies in the evaluation of the equivocation 
at Eve. As a matter of fact, the presence of side information at the eavesdropper, that can be 
used together with its channel output to estimate the source, prevents from directly applying 
secrecy capacity results [3J. We derive a general outer bound on the achievable region, referred 
to as the rate-distortion-equivocation region, for arbitrary channels and side informations. We 
then propose a pure digital scheme which combines secure source coding [fT3l . 031 with coding 
for broadcast channels with confidential messages [0, and derive the corresponding single-letter 
inner bound. These two bounds do not match in general but we derive two results of optimality 
when: (i) Bob has less noisy side information, and (ii) Eve has less noisy channel. In these 
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cases, separation holds and the optimal schemes reduce to a Wyner-Ziv source encoder IfTOl 
followed by a classical wiretap channel encoder J3), and a secure source encoder [fT3l . Ifl4l 
followed by a conventional channel encoder |fT51l , respectively. However, we show through a 
simple counterexample with a binary source that a pure analog scheme can outperform the 
digital one while being optimal. Then, restricting our attention to the matched-bandwidth case, 
we propose a novel hybrid digital/analog scheme that aims to gather the advantages of both 
digital and analog ones, and derive its single-letter inner bound. In the quadratic Gaussian 
setup when side information is only present at the eavesdropper, this strategy is proved to be 
optimal. Furthermore, it outperforms both digital and analog schemes and cannot be achieved 
via time-sharing. We also consider secure transmission of a binary source with BEC/BSC side 
informations over a type-II wiretap channel. The proposed hybrid digital/analog scheme turns 
out to be useful also in this setting. 

The rest of this paper is organized as follows. Section [II] states definitions along with the 
general outer bound on the rate-distortion-equivocation region. Section [III] provides a single- 
letter inner bound based on a digital scheme, as well as special cases where separation holds. 
The proof of the inner bound is given in Section [IVj Transmission of a binary source over a type- 
II wiretap channel is studied in Section [V] providing a counterexample for the optimality of the 
digital scheme. A single-letter inner bound based on a hybrid digital/analog scheme is provided 
in Section ED The proof is given in Section IVIIl Section IVIIII (resp. Section IPXl) presents an 
application example to the transmission of a binary source over a type-II wiretap channel (resp. 
a Gaussian source over a Gaussian wiretap channel with side informations). Section [X] concludes 
the paper. 

Notation 

For any sequence (xj)j eN », notation x£ stands for the collection (x kl x^+x, ■ ■ ■ , x n ). x™ is simply 
denoted by x n . Entropy is denoted by H(-), and mutual information by /(•; ■). We denote typical 
and conditional typical sets by Tg(X) and Tg(Y\x n ), respectively (see Appendix lAl for details). 
Let X, Y and Z be three random variables on some alphabets with probability distribution p. 
If p(x\yz) = p(x\y) for each x,y,z, then they form a Markov chain, which is denoted by 
X -e- Y -e- Z. Random variable Y is said to be less noisy than Z w.r.t. X if I(U ; Y) > I(U ; Z) 
for each random variable U such that U -e- X -e- (Y, Z) form a Markov chain. This relation is 
denoted by Y y x Z. The set of nonnegative real numbers is denoted by R + . For each 
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notation [x} + stands for max{0 ; x}. Logarithms are taken in base 2 and denoted by log(-). The 
binary entropy function is defined on [0, 1] as h 2 (x) = — xlog(x) — (1 — x) log(l — x). Its inverse 
h^ 1 is defined on [0, 1] and takes values in [0, |]. For each a, b G [0, 1], a-kb = a(l — b) + (l — a)b. 
The Bernoulli distribution of parameter u is denoted by B(u). 

II. Problem Definition and General Outer Bound 
A. Problem Definition 

In this section, we give a more rigorous formulation of the context depicted in Fig. Q] Let 
A, B, £, X, y, and Z be six finite sets. Alice, Bob, and Eve observe the sequences of random 
variables (v4j) ieN ., (i^) ieN », and (Ei)^*, respectively, which take values on A, B, and £, resp. 
For each i G W, the random variables Ai, E>i, and Ei are distributed according to the joint 
distribution p(abe) on A x B x £. Moreover, they are independent across time i. Alice can also 
communicate with Bob and Eve through a discrete memoryless channel with input X on X, and 
outputs Y, Z on y, Z, respectively. This channel is defined by its transition probability p(yz\x). 

Let d: A x A — > [0,<i max ] be a finite distortion measure i.e., such that < d max < oo. We 
also denote by d the component-wise mean distortion on A n x A n i.e., for each a n ,b n G A n , 

d(a n ,b n ) = ±Y J n i=id(a l X)- 

Definition 1 (Code): An (n, m)-code for source-channel coding is defined by 

• a (stochastic) encoding function at Alice F: A n — > X m , defined by some transition proba- 
bility P F (x m \a n ), 

• a decoding function at Bob g: B n x y m — > A n . 

The rate of such a code is defined as the number of channel uses per source symbol — . 

Definition 2 {Achiev ability): A tuple (k, D, A) G is said to be achievable if, for any e > 0, 
there exists an (n, m)-code (F, g) such that: 

-<k + e, (1) 

n 

E [d{A n , g{B n , Y m ))] < D + e , (2) 

- H(A n \E n Z m ) > A — £ , (3) 
n 

with channel input X m as the output of the encoder F(A n ). 

The set of all achievable tuples is denoted by 1Z* and is referred to as the rate-distortion- 
equivocation region. 



January 2012 



DRAFT 



SUBMITTED TO THE IEEE TRANSACTIONS ON INFORMATION THEORY 



B' 



A n 



E' 







ym 








DMC 

p(yz\x) 


Bob 








— > Alice > 








Eve 













A n : d(A n , A n ) < D 
±H(A n \E n Z m ) > A 



Figure 1: Secure transmission with side information at the receivers. 



Remark 1: Region 1Z* is closed and convex. 

Remark 2: Quantities involved in Definition [2] only depend on the marginal distributions p(ae), 
p(ab), p{y\x) and p(z\x). The same applies for subsequent results that provide inner and outer 
bounds on 1Z*. 

B. General Outer Bound 

The following theorem gives an outer bound on 1Z* i.e., it defines region 1Z out D 1Z*. 

Theorem 1 (Outer bound): If (k,D,A) is achievable, then there exist random variables U, 
V, Q, T, X on finite sets U, V, Q, T, X, respectively, with joint distribution p[uvqtabexyz) = 
p(uv\a)p{abe)p{q\t)p(t)p(x\t)p(yz\x), and a function A: V x B — > A, verifying the following 
inequalities: 

I(V;A\B)<kI(T;Y) , (4) 
D > E[d(A,A(V,B))] , (5) 
A < H{A\UE) — I(V;A\B) - I{U;A\B) -k(l{T;Y\Q) - I(T;Z\Q)^ . (6) 
Proof: See Appendix O ■ 

III. Digital Scheme 

In this section, we propose a digital coding scheme for secure transmission with side infor- 
mation and derive the corresponding single-letter inner bound "^digital (Theorem [2]). This scheme 
turns out to be optimal under some less-noisy conditions (Propositions \T\ and [2]). 
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A. General Statement 

The following theorem gives an inner bound on 1Z* i.e., it defines region /^digital Q H*- The 
achievability follows by combining secure source coding of lfT3ll . lfT4l with coding for broadcast 
channels with confidential messages [3J. This scheme will be referred to as the digital scheme. 

Theorem 2 (Digital scheme): A tuple (k,D,A) E is achievable if there exist random 
variables U, V, Q, T, X on finite sets U, V, Q, T, X, respectively, with joint distribution 
p(uvqtabexyz) = p(u\v)p(v\a)p(abe)p(q\t)p(t)p(x\t)p(yz\x), and a function A: V x B — > A, 
verifying the following inequalities: 

I(U;A\B)<kI(Q;Y) , (7) 

I(V;A\B)<kI(T;Y) , (8) 

D > E[d(A, A(V, B))] , (9) 



A < H(A\UE) - I(V; A\UB) - k(j(T; Y\Q) - I(T; Z\Q) 



(10) 



Proof: See Section |IVl ■ 
Inequalities ©, ([8]) correspond to sufficient conditions for the transmission of two source 
layers U, V in channel variables Q, T, respectively. The first layer (U, Q) can be seen as a 
common message which is considered to be known at Eve, as shown by the term H(A\UE) in 
(TTOl) . The second layer (V, T) forms a private message which is (partially) protected by adding 
an independent random noise O, Q. The term in square brackets in (flOl) corresponds to the 
information that Eve can still obtain on this protected layer. 

Notice that the inner and outer bounds 7^. d i g i ta i and 1Z 0Ut do not meet in general: 

• Condition © in Theorem [2l which is needed in our scheme to characterize the equivocation 
at Eve, may not be optimal in the general case (see Theorem [TJ). 

• The Markov chain U -e- V -e- A -e- (B, E) is assumed in Theorem |2] while only (U, V) — 
-e- A -e- (B, E) is proved for arbitrary codes in Theorem [TJ 

We provide in Section IIII-CI several cases where 7^ d i g i ta i is optimal. 

B. Coding Scheme Based On "Operational" Separation 

In traditional separated schemes, two stand-alone components successively perform source 
and channel coding, as depicted in Fig. [2l However the scheme that achieves region 72- d i g itai does 
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Figure 2: Traditional separation. 



Figure 3: Proposed system ("operational" sep- 
aration). 



not satisfy this separation principle: The source encoder outputs two layers which are further 
encoded by using the channel code for a broadcast channel with confidential messages [Q (see 
SectionHVl). This results in two independent (but not stand-alone) source and channel components 
yielding statistically independent source and channel variables (as in [20j for Slepian-Wolf coding 
over broadcast channels) i.e., "operational" separation holds (see Fig. [3]). As a matter of fact, 
inequality © in Theorem [2] prevents from separately choosing variables U and Q which would 
maximize the equivocation rate at Eve (fTOl) . 



C. Special Cases 

In this section, we characterize the optimality of the inner bound 72-digitai for some special 
cases. 

1) Bob Has Less Noisy Side Information: If Bob has less noisy side information than Eve 
i.e., B y A E, the optimal coding scheme reduces to a Wyner-Ziv source encoder ifTOll followed 
by a classical wiretap channel encoder [3], and hence separation holds (Fig. [2]): 

Proposition 1: If B y A E, (k,D,A) e is achievable if and only if there exist ran- 
dom variables V, Q, T, X on finite sets V, Q, T, X, respectively, with joint distribution 
p(vqtabexyz) = p(v\a)p(abe)p(q\t)p(t)p(x\t)p(yz\x), and a function A: V x B — > A, verifying 

I(V;A\B)<kI(T;Y) , 

D > E[d(A, A(V, B))] , 

A < H(A\E) - [l(V; A\B) - k(l(T; Y\Q) - I{T; Z\Q) 
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Proof: The above region is achievable by setting variable U to a constant value in Theorem[2] 
On the other hand, the third inequality of Theorem Q] writes: 

A < H(A\UE) 

A < H(A\VB) + I(A; B\U) - I (A; E\U) + k(l(T; Y\Q) - I(T; Z\Q)^ . 

Since B y A E, and U -e- A -e- (B,E) form a Markov chain, 7(A;5|C/) - I(A;E\U) < 
I (A; B) - I (A; E). Moreover H(A\UE) < H{A\E). In this case, the outer bound TZ oat is thus 
included in (and consequently equal to) 7£ d igitai- ■ 

If the informations at Eve (both side information, and channel output) are degraded versions 
of Bob's ones i.e., if both Markov chains A-e-B-e-E, and X-e-Y -e- Z hold, then Proposition Q] 
reduces to the results in [17J. In this case, variable Q is set to a constant value, and T = X. 

2) Eve Has Less Noisy Channel: If Eve has less noisy channel than Bob i.e., Z y x Y, the 
optimal scheme reduces to a secure source encoder [fT3l . |fl4j followed by a conventional channel 
encoder lfT51 . and hence separation holds (Fig. |2]): 

Proposition 2: If Z y x Y, (k, D, A) 6 is achievable if and only if there exist random 
variables U, V, X on finite sets U, V, X, respectively, with joint distribution p(uvabexyz) = 
p(u\v)p(v\a)p(abe) p(x)p(yz\x), and a function A : V x B — > A, verifying 

I(V;A\B) < kI(X;Y) , 

D > E[d(A, A(V,B))] , 

A < H(A\VB) + I (A; B\U) - I (A; E\U) . 

Proof: The above region is achievable by setting Q = T = X in Theorem [2l A new proof 
is needed to obtain the converse part. Here, auxiliary variables are defined as follows, for each 
i E {1, . . . , n}, and each j 6 {1, . . . , m}: 

Ui = ( B? +1) E*-\Y m ) , 

V t = {A l -\B'-\B't^W-\Y m ) , 

Q J = ( E n ,Yi-\Zf +l ) , 

Tj = (A n ,E n ,Yi-\Z™ +1 ) . 
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Now, both Ui -e- V { -e- A { -e- Ej), and -e- 7j -e- X,- -e- (Y}, form Markov chains 
(see Fig. [T5l) . Following the arguments given in Appendix we can define new variables U, 
V, Q, T verifying the above Markov chains and the following inequalities: 

I(V;A\B)<kI(T;Y) , 
D > E[d(A, A(V,B))] , 

A < H(A\UE) - I(V; A\UB) + k(j{T; Y\Q) - I(T; Z|Q)) . 

Since Z hx Y, and Q -e- T -e- X -e- (Y, Z) form a Markov chain, J(T; F|Q) - /(T; Z|Q) < 
and J(T; Y) < I(X; Y). This concludes the proof. ■ 

3) Secure Source Coding: Defining the transmitted rate as R = kI(X;Y), Proposition [2] 
provides the single-letter characterization of the rate-distortion-equivocation region in the setup 
of secure source coding with uncoded side information given in 031 Theorem 3]. 

4) Wiretap Channel: Choosing appropriate side informations and auxiliary variables, region 
^digital reduces to the achievable region for the wiretap channel Q Eq. (2.6)]. 

IV. Proof of Theorem [2] (Digital Scheme) 

Let U, V, Q, X be four random variables on finite sets U, V, Q, X, respectively, such 
that piuvqabexyz) = p(u\v)p(v\a)p(abe) p(q\x)p(x)p(yz\x), a function A: V x B — > A, and 
a tuple (k,D,A) 6 E^_. In this section, we describe a scheme that achieves (under some 
sufficient conditions) tuple (k,D,A) i.e., for any e > 0, we construct an (n,m)-code (F,g) 
that verifies ©-®. 

In this scheme, Alice compresses the source A in descriptions (U, V), with V on the top of 
U . In view of the side information at Bob B, random binning a la Wyner-Ziv ifTOll is performed. 
The corresponding bin indices (r 1; r 2 ) are then mapped to indices (r c ,r p ), which are further 
transmitted to Bob through variables (Q, X) using a code for broadcast channel with confidential 
messages 0, where index r p is protected by an independent random noise 77. As in the classical 
wiretap channel 0, Q, its rate Rf satisfies some constraint that allows to characterize the 
equivocation rate at Eve. 

Let e > 0, Ri, R 2 , R c , R p , R f E M* + , Si > R u S 2 > R 2 such that 

R f <(k + e) I(X; Z\Q) , (11) 
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v n (si,s 2 ) 



2n(S , 2 -R 2 ) 



< t 2 nR2 

2'nRi 

Figure 4: Digital scheme-Source codebook. 

and assume that a local (independent and uniformly distributed) random source with rate Rf is 
available at Alice. Define 7 = 77^. 

A. Codebook Generation 

1) Source Codewords: Randomly pick 2 nSl sequences u n (si) from Tg(U) and divide them 
into 2 nRl equal size bins Bi(n), r\ G {1, . . . , 2 nRl }. Then, for each codeword u n (si), randomly 
pick 2 nS2 sequences f n (si,s 2 ) from Tg(V\u n (si)) and divide them into 2 nR2 equal size bins 
£ 2 (si, r 2)' t-2 G {1, ... , 2 nR2 }. See Fig. H 

2) Channel Codewords: Randomly pick 2 nRc sequences q m (r c ) from T™(Q). Then, for each 
codeword q m (r c ), randomly pick 2 n ( Rp+R ^ sequences x m (r c ,r p ,rf) from Tp(X\q m (r c )). See 
Fig. El 

B. Encoding Procedure 

Assume that source sequence A n and random noise 77 are produced at Alice. 

Look for the first codeword u n (s\) such that (u n (si), A n ) 6 Tg(U, A). Then look for a 
codeword f n (si,s 2 ) such that (v n (s 1 ,s 2 ),A n ) e T r 5 l (V, A\u n (si)). Let B x {ri) and S 2 (si,r 2 ) be 
the bins of u n (si) and f n (si, s 2 ), respectively. 

Define (r c , r p ) = M(r 1; r 2 ) G {1, . . . , 2 nRc } x {1, . . . , 2 nRp } where M is an arbitrary fixed 
one-to-one mapping such that n = M'(r c ) for some mapping M'. These two functions can be 
defined if: 

Ri + R2 = R c + R p , (12) 
R 1 <R C . (13) 



u n ( Sl ) 



2n{Si-Ri) 
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Figure 5: Digital scheme-Channel codebook. 

Alice then sends X m = F(A n ) = x m (r c ,r p ,rf). 

C. Decoding Procedure 

Assume that Bob observes B n and receives Y m from Alice. 

Look for the unique codeword q m (r c ) such that (q m (r c ), Y m ) G T™(Q, Y). Then look for the 
unique codeword x m (r c ,r p ,rf) such that (x m (r c , r p , 77), Y m ) G T™(X, Y|g m (r c )). 
Compute (r 1; r 2 ) = M -1 (r c , r p ). 

Look for the unique codeword u n (si) G -Bi( r i) such that (w n (si), £> n ) G T^U, B). Then look 
for the unique codeword f™(si,s 2 ) G B 2 (si,r 2 ) such that (u n (si, s 2 ), S n ) G ^(V, B\u n (si)). 

Compute the estimate g(B n , Y m ) G A n using the following component-wise relation, for each 
i = {1, . . . ,n}: 

ft(s n ym) A^. (si)S2))jB .) 

D. Errors and Constraints 

Denoting by E the event "An error occurred during the encoding or decoding steps," we 
expand its probability (averaged over the set of all possible codebooks) as follows: Pr{E} < 
P t 1 + P ty 2 + P e ,x + P e ,2 + Pd.i + Pd,2 + Pd,3 + Pd,4, where each term corresponds to a particular 
error event, as detailed below. We derive sufficient conditions on the parameters that make each 
of these probabilities small for some sufficiently large n. From now on, let m = [n(k + 

'Note that m — > oo as n — > oo. 
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1 ) Typicality: From standard properties of typical sequences (see Appendix |A]), there exists 

a sequence rj n ► such that P tl = Pr {(A r \ B r \ E n ) G" T£(A,B,E)} < r] n . Consequently, 

Pt,i < 7 f° r some sufficiently large n. 

Similarly, since the input of the channel X m is set to some codeword x m (r c , r p , r/) G Tj Ti (X), 
P t 2 4 p r {(X m , F m , Z m ) Tf (X, y, Z)} < 7 for some sufficiently large n. 

2) Encoding: In the first encoding step, Alice needs to find (at least) one codeword u n (s\) 
such that (u n (si), A n ) e Tg(U,A). Following standard argument (see e.g. |fl4l Appendix B-F- 
2]), we can prove that if Si > I(U ; A), then the probability that this step fails P e> i can be upper 
bounded by 7 for some sufficiently large n. 

Similarly, the second encoding step succeeds with probability 1 — P e 2 > 1 — 7 under condition 
S 2 > I(V;A\U). 

3) Decoding Indices: In the first decoding step, Bob looks for the unique codeword q m (r c ) 
such that (q m (r c ),Y m ) 6 T™(Q,Y). Following standard argument for channel coding, we can 
prove that if R c < (k + e)I(Q;Y), then the probability that there exists another admissible 
codeword P^i can be lowered below 7 for some sufficiently large n. 

Similarly, the second encoding step succeeds with probability 1 — P^ 2 > 1 — 7 under condition 
R p + R f < (k + e)I(X;Y\Q). 

4) Decoding Source Variables: In the third decoding step, Bob looks for the unique codeword 
u n (si) G Bi(n) such that (u n (si), B n ) G T^(U,B). Following standard argument for source 
coding (see e.g. lfT4l Appendix B-F-4]), we can prove that if S\ — R\ < I(U;B), then the 
probability that there exists another admissible codeword P d>3 can be lowered below 7 for some 
sufficiently large n. 

Similarly, the fourth decoding step succeeds with probability 1 — P d 4 > 1 — 7 under condition 

S 2 -R 2 <I{V;B\U). 

5) Summary: In this paragraph, we proved that under some sufficient conditions, Pr {E} < 87. 
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E. Distortion at Bob 

We now check that our code achieves the required distortion level at Bob (averaged over the 
set of all possible codebooks): 



E 



d{A n , g(B n , Y m ))\ < (1 - Pr {E})E [d(A n ,A(v n (s h s 2 ), B n )) t\ + Pr {E} d n 

e 8e 



< E[d(A, A(V, B) n () () 

where the last inequality holds for some sufficiently large n, and follows from Pr{E} < 
87, the definition of 7, and standard argument in rate-distortion theory from the fact that 
(A n ,v n (s 1 ,s 2 ),B n ) G Tf(A,V,B) when no error occured (see e.g. d Appendix B-G]). 
Condition D > E[d(A, A(V, -B))] is thus sufficient to achieve distortion D + e at Bob. 

F. Equivocation Rate at Eve 

In the following paragraphs, we prove a lower bound on the equivocation rate at Eve. We first 
split up the equivocation into two terms which will be studied separately: 

H(A n \E n Z m ) = H(A n \r c r p E n Z m ) + I(A n ; r c r p \E n Z m ) . (14) 

v v ' v v ' 

E s E c 

1) Study of E s : The "source" term E s writes: 

E s = H{A n \ ri r 2 E n ) 

= H{A n \ ri E n ) - H{r 2 \ ri E n ) 

= H(A n \ ri E n ) - H(r 2 ) + I(r 2 ; r x E n ) 

> nH(A\UE) -n\-nR 2 + J(r 2 ; E n \r x ) , (15) 

where 

• step (a) follows from the Markov chain A n -e- (r c , r p , E n ) -e- Z m and the identity (r c , r p ) = 
M(ri,r 2 ) where M is a one-to-one mapping, 

• step (b) from the fact that the bin index r 2 is a deterministic function of A n , 

• step (c) from the fact that the codewords M n (sx) are drawn i.i.d., for some sufficiently 
large n (see HH Lemma 6]), the fact that r 2 G {1, . . . ,2 ni?2 }, and the non-negativity of 
mutual information. 

Note that this term corresponds to the one studied in lfl4l Appendix B-H]. The above lower 
bound should however be tighter since we do not neglect the remainder term I(r 2 ; E n \ri). 
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2) Study of E c : The "channel" term E c writes: 



E c = H{r c r p \E n Z m ) 

= H(r p \r c Z m ) + H(r c \Z m ) - I(r c r p ; E n \Z m ) , (16) 

where the first step follows from the fact that (r c , r p ) is a deterministic function of A n . 

The first term of the r.h.s. of (fT6l) corresponds to the equivocation (of the private message, given 
the common message and the output of the channel) in the wiretap channel setting. Following 
the arguments of (31 Section IV], 0, Section 2.3]a we can easily prove the following lower 
bound: 

H{r p \r c Z m ) > n{R p + R f ) - mI(X; Z\Q) - 1 - n § , (17) 

for some sufficiently large m. 

3) End of Proof: Gathering (fl4)) - (fT7l) . we proved that: 

H(A n \E n Z m ) > nH(A\UE) - nR 2 + n(R p + Rf) - mI(X; Z\Q) 

+ J(r 2 ; E n \n) + H(r c \Z m ) - I(r c r p ; E n \Z m ) - n f - 1 . (18) 

We now study the remainder of the r.h.s. of the above inequality: 

J(r 2 ; E n \n) + H(r c \Z m ) - I(r c r p , E n \Z m ) 

= /( ri r 2 ; E n ) - J(n; E n ) + H(r c \Z m ) - I(Z m r c r p , E n ) + I(Z m ; E n ) 
= - /(n; E n ) + H(r c \Z m ) + I(Z m - E n \r c r p ) + I(Z m - E n ) 

> - J(n; E n ) + I{r c Z m ; E n ) + ^(rel^Z 7 ") 

(c) 

> , 

where 

• step (a) follows from the identity (r c , r p ) = M(ri,r 2 ) where M is a one-to-one mapping, 

• step (6) from the non-negativity of conditional mutual information, 

2 The proof relies on (i) definition X m — x m (r c ,r p ,rf), (ii) the fact that codewords x m (r c ,r p ,rf) are nearly uniformly 
distributed (given r c ) over a set of cardinality 2 n ^ Rp+R f \ (Hi) the fact that the channel X h-> Z is memoryless, (7v) Fano's 
inequality together with constraint i ll It . which ensures that Eve can decode x m (r c , r p , r/) from (r p ,rf) with an arbitrarily 
small probability of error, (v) standard properties of typical sequences, and (v;) the Markov chain Q -e- X -e- Z. 
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• step (c) from the fact that ri = M'(r c ) for some mapping M', and the non-negativity of 

conditional entropy and mutual information. 
Inequality (TT8T) then yields 



I 777 

-H(A n \E n Z m ) > H(A\UE) -R 2 + R p + R f I(X; Z\Q) - e , 

n n 

for some sufficiently large n. 

Condition A < H(A\UE) - R 2 + R p + R f - (k + e)I(X; Z\Q) is thus sufficient to achieve 

equivocation rate A — e. 



G. Summary of Sufficient Conditions 

Putting all pieces together, we proved that the following inequalities are sufficient conditions 
for a tuple (k, D, A) G IR^ to be achievable: For each e > 0, 

Ri, R 2 , R c , R P , Rf > 

Ri + R 2 = R c + Rp 
Ri < R c 

51 > Ri 

52 > R2 
S 1 >I{U-A) 
S 2 > I(V;A\U) 
R c < (k + e)I(Q;Y) 

R p + R f < (k + e)I(X;Y\Q) 

5 1 - R 1 < I(U; B) 

5 2 - R 2 <I{V;B\U) 
R f < (k + e)I(X;Z\Q) 

D > E[d(A, A(V, B))] 

A < H(A\UE) -R 2 + R p + R f -(k + e)I(X; Z\Q) 
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Using Fourier- Motzkin elimination [26], it is straightforward to prove that this system of 
inequalities is equivalent to: 

'I(U;A\B) < (k + e)I{Q;Y) 

I(V;A\B) < (k + e)I(X]Y) 

< D > E [d(A, A(V, B))] 
A < H(A\UE) 

A < H(A\UE) - I(V; A\UB) + (k + e) (l(X; Y\Q) - /(X; Z|Q)) 
H. Channel Prefixing 

For each random variable T on some finite set T such that T -e- X -e- (Y, Z) form a Markov 
chain, we can use the above scheme considering the DMC T i— > (Y, Z) instead of X i— > (Y, Z). 
In this case, the above sufficient conditions write 

'I(U;A\B) < {k + e)I{Q;Y) 

I(V;A\B) < (k + e)I(T;Y) 

< D > E[d(A, A(V, B))] 
A < H{A\UE) 

A < H(A\UE) - /(V; A|C7B) + (k + e) (l(T; Y\Q) - J(T; Z|Q)) 
Since region 72.* is closed, this proves Theorem [2l ■ 

V. Secure Transmission of a Binary Source With BEC/BSC Side Informations 

Over a Type-II Wiretap Channel 

A. System Model 

Consider the source model depicted in Fig. [6l where the source is binary uniformly distributed 
(A ~ B (I)) and the side information at Bob, resp. Eve, is the output of a binary erasure channel 
(BEC) with erasure probability (3 E (0, 1], resp. a binary symmetric channel (BSC) with crossover 
probability e G [0, |], with input A. 

Recall that according to the values of the parameters (j3,e), the side informations satisfy the 
properties summarized in Fig. [7] lETl . 
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1-/3 



1 - e 



Figure 6: Binary source with BEC/BSC side informations. 







2e 



4e(l - e) 



Me) 



ft 



A -e- B -e- _E B) > J(A; J5) 

Figure 7: Relative properties of the side informations as a function of (/3,e). 

The communication channel is similar to the type-II wiretap channel of O: It consists of a 
noiseless channel from Alice to Bob, and a BSC with crossover probability £ £ [0, |], from 
Alice to Eve (see Fig. [8]). 

In this section, we focus on lossless reconstruction at Bob (d is the Hamming distance and 
D — 0) and matched bandwidth (k = 1). 



5. Performance of Coding Schemes 

From the general outer bound of Theorem [Q we can easily derive the following result. 
Proposition 3 (Outer bound): If (k = 1, D = 0, A) is achievable, then there exist u, q £ [0, |] 
such that 

A</i 2 (e) + /i 2 («)-/i 2 (e*«)- [^(u)- (MO + W-MC*?))] • 

Proof: The proof is similar to the one of the converse part of Proposition @] below, given 
in Appendix [Ql Details are omitted. ■ 
The following proposition provides a simple expression of region T^digitai- 
Proposition 4 (Digital scheme): (k = 1, D = 0, A) £ 72-digitai if an d only if there exist u,q £ 
[0, |] such that 

0{l-h 2 (u)) <l-h 2 {q) , 



A < Me) + Mu) - h 2 (e*u) 



Ph 2 (u)-(h 2 {0 + h 2 {q)-h 2 ((*q) 
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Proof: The proof of the converse part is given in Appendix [Dj 
The direct part follows from Theorem [2] after some straightforward manipulations choosing 
auxiliary variables as follows (details are omitted): V = A; X ~ <^(|); U (resp. Q) is the 
output of a BSC with crossover probability u 6 [0, ~] (resp. q £ [0, -]) and input /I (resp. X). 

■ 

Notice that if (3 < 4e(l — e), then B ^ A E (see Fig. [7]), and hence Proposition \T\ holds i.e., 
the above inner bound is optimal and separation holds. 

In the following, we will compare the above digital scheme with a pure analog one, consisting 
in directly sending the source over the channel. Its performance is given by the following 
proposition. 

Proposition 5 (Analog scheme): A tuple (k = 1, D = 0, A) £ is achievable through an 
analog scheme if 

&<h 2 (e) + h 2 (C)-h 2 ((*e) . 

Proof: Letting X = A yields zero distortion at Bob (since Y = X) and equivocation 
rate H(A\EZ) at Eve. The above expression follows after some straightforward manipulations. 
Details are omitted. ■ 

C. Counterexample for the Optimality of Theorem \2\ 

Let now assume that Bob does not have any side information i.e., B = or equivalently 
(3 = 1, and let e = ( = 0.1, so that A -e- E -e- B and X -e- Y -e- Z form Markov chains, and 
neither Proposition [Q nor Proposition [2] applies. 

This setting provides a counterexample for the general optimality of the inner bound in 
Theorem 12 Numerical optimization over u and q in Proposition |4] indicates that the proposed 
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digital scheme achieves an equivocation rate A = 0.056, while the naive analog scheme of 
Proposition [5] achieves A = 0.258. Furthermore, the latter coincides with the outer bound of 
Proposition [3j This shows that a joint source-channel scheme may achieve better performance 
in some cases. 

VI. Hybrid Coding 

Based on the observations of the previous section about the usefulness of analog schemes, 
we propose in this section a hybrid digital/analog scheme that yields a new single-letter inner 
bound 7?.hybrid hi the matched-bandwidth case (Theorem [3]). 

A. General Statement 

Channels A i->- B and X \-t Y can be viewed together as a state-dependent channel with input 
X, state A and output (B, Y). In this perspective, Alice and Bob form a communication system 
with channel state information non-causally known at the transmitter (CSIT), as depicted in 
Fig. HI Roughly speaking, the proposed scheme consists in sending independent digital random 
noise r f using a Gelfand-Pinsker code [ 281 for this equivalent state-dependent channel. 

Theorem 3 (Hybrid scheme): A tuple (k = 1, D, A) e is achievable if there exist random 
variables U, V, X on finite sets U, V, X, with joint distribution p(uvabexyz) = p(u\v)p(vx\a) 



p(abe)p(yz\x), x = x(v, a), and a function A: V x B x y — > A, verifying 

I{U ; A) < I(U ; BY) , (19) 

I{V;A\U)<I(V;BY\U) , (20) 

D >E [d(A, A(V, B, Y))] , (21) 

A < H{A\UE) - I(V; A\U) - I(X; Z\UE) + min {/(V; BY\U) ; J(V; AZ\U)} . (22) 

Proof: See Section lYUI ■ 



Inequalities ( TT9l , (1201) correspond to sufficient conditions for the transmission of descriptions 
U, V of A The first layer U can be seen as a common message which is considered to be known 
at Eve, as shown by the term H(A\UE) in (1221) . Digital random noise rj helps to secure the 
second layer V against Eve. 
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p{b\a) 


B n 

> 


Bob 




p(y\x) 




i Y n 



f f , A n ) 



equivalent state-dependent channel 
Figure 9: Alice and Bob as a system with state-dependent channel and CSIT. 



B. Special Cases 

1) Analog Schemes: The proposed scheme can reduce to a pure analog one (as the simple 
one of Proposition [5]). Hence, 7£h y brid contains tuples that may not be in 72. d i g itai : ^hybrid <t- "^-digital- 

2) Digital Schemes: By defining the variables in Theorem [3] as pairs of independent source 
and channel components, we can obtain the structure of those in Theorem [21 but such variables 
do not verify all inequalities and thus 7^ d i g itai <t- ^hybrid- 

3) Wiretap Channel: Choosing independent source and channel variables with appropriate 
rates, region 7£h y brid reduces to the achievable region for the wiretap channel flTJ Eq. (2.6)]. 

VII. Proof of Theorem [3] (Hybrid Scheme) 

Let U, V, X be three random variables on finite sets U, V, X, respectively, such that 
p(uvabexyz) = p(u\v)p(vx\a)p(abe)p(yz\x), x = x(v,a), a function A: V x B x y — > A, 
and a tuple (D,A) G E^_. In this section, we describe a scheme that achieves (under some 
sufficient conditions) tuple (k = 1,D,A) i.e., for any e > 0, we construct an (n, n)-code (F,g) 
such that: 

E[d(A n ,g{B n ,Y n ))] <D + e , 

-H(A n \E n Z n ) > A-e . 
n 

In this scheme, Alice compresses the source A in descriptions (U, V), with V on the top of 
U . Digital random noise r/ is also transmitted on V (a la Gelfand-Pinsker) to take advantage of 
the possibly better quality of Bob's channel, and prevent Eve from decoding the whole message. 
As in the classical wiretap channel Q, Q, its rate Rf satisfies some constraint that allows to 
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characterize the equivocation rate at Eve. Alice finally sends some deterministic function x(V, A) 
of V and A. 

Let e > 0, R u R 2 , R f G JR+ such that 

R 2 +R f <I{V;AZ\U) , (23) 

and assume that a local (independent and uniformly distributed) random source with rate Rf is 
available at Alice. Define 7 = ■=-§— . 

A. Codebook Generation 

Randomly pick 2 nRl sequences u n (ri) from T£{U). Then, for each codeword u n (ri), randomly 
pick 2 n{ ~ R2+R ^ sequences v n (r 1 ,r 2 ,rf) from Tg-(V\u n (ri)). See Fig. [TOl 

B. Encoding Procedure 

Assume that source sequence A n and random noise r/ are produced at Alice. 

Look for the first codeword u n (ri) such that (u n (ri), A n ) G T^(U, A). Then look for the first 
codeword v n (r 1 ,r 2 ,rf) such that (v"(r 1 ,r 2 ,rf), A n ) G Tg(V, A\u n (r{)). 

Alice then sends X n = F(A n ) defined by the following component-wise relation, for each 
i = {l,...,n}: 

Xi = x(vi(ri,r2,rf),Ai) . 

C. Decoding Procedure 

Assume that Bob observes B n and receives Y n from Alice. 



u n (rx) 



v n (n,r 2 , rf) 



2 nR f 



Figure 10: Hybrid scheme-Codebook. 
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Look for the unique codeword u n (ri) such that (u n (ri), B n , Y n ) e Tg(U, B, Y). Then look 
for the unique v n (r 1 , r 2 , rf) such that (v n (r 1 , r 2 , r/), B n , Y n ) G T^(V, B,Y\u n (ri)). 

Compute the estimate g(B n , Y n ) E A n using the following component- wise relation, for each 
i = {l,...,n}: 

9i {B n ,Y n ) ± Aiviin^rf)^^) . 

D. Errors and Constraints 

Denoting by E the event "An error occurred during the encoding or decoding steps," we expand 
its probability (averaged over the set of all possible codebooks) as follows: 
Pr{E} < P t i + P t2 + P e i + P e ,2 + Pd,i + Pd,2, where each term corresponds to a particular 
error event, as detailed below. We derive sufficient conditions on the parameters that make each 
of these probabilities small. 

1 ) Typicality: From standard properties of typical sequences (see Appendix [A]), there exists 
a sequence rj n ► such that P tl = Pr {(A n , B n , E n ) T^A, B, E)} < r] n . Consequently, 

n— too 

Pt,i < 7 f° r some sufficiently large n. 

Similarly, since the input of the channel X n is typical (when no error occurs during the 
encoding steps), P tj2 = Pr {(X n , Y n , Z n ) £ T 5 n (X, Y, Z)} < 7 for some sufficiently large n. 

2) Encoding: In the first encoding step, Alice needs to find (at least) one codeword u n (ri) 
such that (w n (ri), A n ) e Tg(U, A). Following standard argument (see e.g. [|T4l Appendix B-F-2], 
we can prove that if Ri > I(U;A), then the probability that this step fails P 6]1 can be upper 
bounded by 7 for some sufficiently large n. 

Similarly, the second encoding step succeeds with probability 1 — P e 2 > 1 — 7 under condition 
R 2 > I(V;A\U). 

3) Decoding: In the first decoding step, Alice should find the unique codeword u n (ri) such 
that (u n (ri), B n , Y n ) 6 Tg(U, B, Y). The corresponding error probability P^i must be carefully 
handled. 

As previously noted in 11241 . [|29l , the conventional random coding proof technique does not 
apply here. In the proposed joint coding scheme, a single codebook plays both roles of source 
and channel codebooks. For a given source sequence a n , the indices (ri,r 2 ) thus depend on the 
entire codebooks, and the averaging over the set of all possible codebooks cannot be performed 
in the usual way. Similarly to (|24|. we can prove the following lemma. 
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Lemma 1: There exists k < 1 and a sequence rj n > such that 

n— >oo 

nn(R 1 -I(U;BY)+ Vn ) 
p <- 

^ ~ (1 - Kf 

for some sufficiently large n. 

Proof: See Appendix El ■ 

From Lemma [H if R 1 < I(U; BY), then probability P d i vanishes as n tends to infinity, and 
hence can be upper bounded by 7 for some sufficiently large n. 

Using similar arguments, we can prove that the second decoding step succeeds with probability 
1 - Pd,2 > 1 - 7 under condition R 2 + R f < I(V; BY\U). 

4) Summary: In this paragraph, we proved that under some sufficient conditions, Pr {E} < 67. 

E. Distortion at Bob 

We now check that our code achieves the required distortion level at Bob (averaged over the 
set of all possible codebooks): 

E [d(A n , g(B n , K n ))j < (1 - Pr {E})E [d(A n , A{v n (r u r 2 ,r f ), B n , Y n ))\t] + Pr {E} d max 

<E[d(A,A(V,B,Y))]+~ + j, 

where the last inequality holds for some sufficiently large n, and follows from Pr{E} < 
67, the definition of 7, and standard argument in rate-distortion theory from the fact that 

(A n , v n (ri, r 2 , 77), B n , Y n ) e T$(A, V, B, Y) when no error occured (see e.g. |[T4l Appendix B- 
G]). 

Condition D > E[d(A, A(V, B, F))] is thus sufficient to achieve distortion D + e. 

F. Equivocation Rate at Eve 

The equivocation at Eve can be divided in "source" and "channel" terms. Each one is studied 
using standard properties of typical sequences, and following the arguments of [[141 Appendix B- 
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H] and Q Section 2. 3. Step 3]: 

H(A n \E n Z n ) = H( ri r 2 r f A n X n \E n Z n )- H( ri r 2 r f X n \A n E n Z r 
{ = } H(r 1 r 2 r f A n X n \E n Z n ) - H(r 2 r f \r 1 A n Z n ) 



4 

(c) 

' - - ■ ■ — £ 

1 



> H(r 1 r 2 r f A n X n \E n Z n ) - n 



> H(r 2 r f A n X n \r l E n Z n ) - n ■ 



= H(r 2 r f A n X n \r l ) + H(E n Z n \r 1 r 2 r f A n X n ) - H(E n Z n \n) -n\ 
( = } ^"(r/^lri) + H(E n Z n \A n X n ) - H(E n Z n \r x ) -n\ 
( = } H{A n \n) + H(r f ) + ^(^"lA") + H(Z n \X n ) 

-E{E n Z n \r x )-n \ , (24) 

where 

• step (a) follows from the fact that X n (resp. r\) is a deterministic function of (n, r 2 , r/, A") 
(resp. A n ), and the Markov chain (r 2 , r/) -e- (A n , Z n ) -e- E n , 

• step (6) from condition (T23T) (which ensures that Eve can decode v n (ri,r 2 , r/) from (« n (ri), 
A n , Z n ) with an arbitrarily small probability of error, using a decoder similar to Bob's one- 
see Sections IVII-Ci IVII-DI) . and Fano's inequality (for some sufficiently large n), 

• step (c) from the fact that conditioning reduces the entropy, 

• step (d) from the fact that (r 2 ,X n ) is a deterministic function of (ri,r/, A n ), and the 
Markov chain (E n , Z n ) -o- (A n ,X n ) -o- {r u r 2 ,r f ), 

• step (e) from the fact that 77 is independent of (ri,A n ), and the Markov chains E n -e- 
A n -o- (X n , Z n ), Z n -e- X™ -e- E n ). 

We now separately study each term of the r.h.s. of (T24l) : 

• From the fact that the codewords u n (rx) are drawn i.i.d., and following the argument of 
ffT4l Lemma 6], we can prove that 

H(A n \ ri )>n(H{A\U)-l) , 

for some sufficiently large n. 

• Since the random source rj is uniformly distributed with rate Rf. 

H(r f ) = nR f . 
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• Since the sources are i.i.d.: 

H(E n \A n ) = nH(E\A) . 

• Since the channel is memoryless and the input X n is typical (see [7, Eq. (2.46)]), 

H(Z n \X n ) >n(H{Z\X) - |) . 

• From the fact that (w n (ri), E n , Z n ) are jointly typical, and following the arguments of (TJ 
Eq. (2.54)], we can prove that 

H(E n Z n \r 1 ) < n (H(EZ\U) + §) . 

Gathering all the above equations, we proved that 

H(A n \E n Z n ) > n(H(A\U) + R f + H(E\A) + H(Z\X) - H(EZ\U) - , 

for some sufficiently large n. 

After some algebraic manipulations using the Markov chains U -e-A-e-E and (U, E)-e-X -e-Z, 
we proved that the following condition is sufficient to achieve equivocation rate A — e at Eve: 

A < H{A\UE) - /(X; Z\UE) + R f . 

G. End of Proof 

In this section, we proved that sufficient conditions for the achievability of a tuple {k = 
1, D, A) are given by the following system of inequalities: 

Ri > /([/; A) 

R 2 > I(V;A\U) 
R f >0 

Ri < I(U;BY) 
R 2 + Rf < I(V;BY\U) 
R 2 + Rf < I(V;AZ\U) 

D > E[d(A,A(V,B,Y))] 
A<H(A\UE)-I(X;Z\UE) + R f 
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Fourier-Motzkin elimination ||26l then yields: 

' I(U; A) < I(U;BY) 

I(V;A\U) < I(V;BY\U) 

D > E[d(A, A(V,B, Y))] 

A < H(A\UE) - I(X; Z\UE) + I(V; BY\U) - I(V; A\U) 

A < H{A\UE) - I(X; Z\UE) + I(V; AZ\U) - J(V; A\U) 
This proves Theorem [3l ■ 

VIII. Secure Transmission of a Binary Source With BEC/BSC Side 
Informations Over a Type-II Wiretap Channel (Continued) 

In this section, we go back on the binary example introduced in Section [V] and compare a 
hybrid coding scheme based on Theorem |3] with the ones analyzed is Section [V] namely the 
digital scheme of Section HIT] (see Proposition |4j) and a pure analog one consisting in directly 
sending the source over the channel (see Proposition 0. 

A. Hybrid Coding 

We consider the hybrid scheme of Theorem |3] choosing variables U, V and X as follows: 

U = V © W , (25) 
V^B^), (26) 
X = V © A , (27) 

where © stands for the binary exclusive-or operator, W is independent of A and V, and W ~ 
B(u) for some crossover probability u e [0, |]. 

5. Numerical Results 

Fig. Qj] represents the largest equivocation rate at Eve A as a function of the erasure proba- 
bility (3 for 

1) the outer bound of Proposition [31 

2) the hybrid digital/analog scheme of Theorem [3] with variables (|25T)-(T27T) (and optimizing 
over u), 
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3) the digital scheme of Proposition [4] (optimizing over u and q), 

4) the analog scheme of Proposition [51 
for parameter values e = 0.1, ( = 0.1. 

If {3 < 4e(l — e), B is less noisy than E (see Fig. [7]), and the digital scheme is optimal (as 
stated by Proposition [B, as well as the proposed hybrid one. Here, this result also seems to hold 
when B is only more capable than E i.e., for f3 < /12(e). 

For f3 — 1, as already noted in Section IV-Cl the naive pure analog scheme outperforms the 
digital one. According to the comments of Section IVI-Bll the proposed hybrid digital/analog 
scheme always performs as good as the analog one. 

In Fig. [TTJ the proposed hybrid digital/analog scheme also seems to perform as good as 
the digital one. However, according to the comments of Section IVI-B21 and depending on the 
parameters e, (, this may not be the case for all values f3 in [/12(e), 1). 



< 0.3 - 




Outer bound (Prop. [3]) 
Hybrid (Th. 
Digital (Prop, g]) 
Analog (Prop. [5]) 



Figure 11: Equivocation rate A as a function of erasure probability (3 (e = 0.1, £ = 0.1). 



IX. Secure Transmission of a Gaussian Source Over a Gaussian Wiretap 

Channel 

A. System Model 

In this section, we consider the transmission of a Gaussian source over a Gaussian wiretap 
channel with matched bandwidth. More precisely, as depicted in Fig. [121 the source at Alice A 
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A/»^ d -AA(0,P B ) — (+) 



JVg^ 4 Af(0,P E ) 



Alice 



yn 



Bob 



Eve 



A 



E' 



Figure 12: Transmission of a Gaussian source over a Gaussian wiretap channel with side 
information. 



is standard Gaussian, and observations at Bob and Eve are the outputs of independent additive 
white Gaussian noise (AWGN) channels with input A and respective noise powers P B and P E . 
Communication channels from Alice to Bob and Charlie are AWGN channels with respective 
noise powers P Y and Pz- The average input power of this channel is limited to P. One channel 
use is allowed per source symbol. 

Euclidean distance on R is used to measure distortion at Bob (d(a, b) = (a — b) 2 , for each 
a, b E R). Differential entropy h(-) measures uncertainty yielding equivocation rates A 6 R. 
We also introduce quantity De = 2 2A /(27re), which provides a lower bound on the minimum 
mean-square error of any estimator of A at Eve [30, Theorem 8.6.6]. 

Definition 3 (Achiev ability): In this section, a tuple (D, De) G R*j_ 2 is said to be achievable 
if, for any e > 0, there exists an (n,n)-code (F, g) such that: 



E[\\A n - g(B n ,Y n )f] <D + e 



- h(A n \E n Z n ) > - log (2vre D E ) - e 
n 2 



i=l 



< P 



with channel input X n as the output of the encoder F(A n ). 



B. First Results 

Although Theorems [T} [2l [3] are stated and proved for finite alphabets, we take the liberty to 
use their statements as inner/outer regions also for this quadratic Gaussian case. The involved 
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probability distributions should now also verify condition 

Var [X] < P . (28) 

The corresponding regions will be denoted with an additional - p i.e., TZ^uv ^-digital an ^ ^hybrid- 
Notice that due to the Gaussian additive noises, and depending on the relative values of Pb, Pe 
(resp. Py, Pz), one side information (resp. one channel) is a stochastically degraded version of 
the other. There exist four different cases and, from the results of Section IIII-CL separation holds 
for three of them, as summarized in Table IB In these cases, we can moreover prove closed-from 
expressions for the achievable region, as stated by Propositions [6] and [7] below. 

Proposition 6: Assume that P B < P E . A tuple (D,D E ) is achievable if and only if 

1 1 



D > 



D R < 1 



1 + — ' 

1 ^ Pe 



1 ( 1 i _P_ 

Py 



1 + 4- 1 



D E < — -D-maxll. ^ 

l+ Pe I 1 + Pz 

Proof: The proof of the converse part is given in Appendix [0 

The direct part follows from Proposition Q] by choosing appropriate Gaussian auxiliary random 
variables. Details are omitted. ■ 

Proposition 7: Assume that P B > P E and Py > Pz- A tuple (D,D E ) is achievable if and 
only if 

D> 1 



1 + — 1 + ' 

1 ^ Pb 1 ^ Py 



Dv < 1 



1 + — ' 

L ^ Pe 



D e < 1 



— + — 1 - 

D Pe Pb 



Proof: The proof of the converse part is given in Appendix [Gl 

The direct part follows from Proposition [2] by choosing appropriate Gaussian auxiliary random 
variables. Details are omitted. ■ 

As a matter of fact, the case when Bob has "better" channel (P Y < P z ) and "worse" side 
information (Pb > Pe) than Eve is still open. In this quadratic Gaussian example, the outer 
bound TZ^ ut reduces to the one given by Proposition [8] below. 
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Pb < Pe 


Pb > Pe 


Py < Pz 


/ 


? 


Py > Pz 


/ 


/ 









Table I: Cases where "/^digital is tight and separation holds. 



Proposition 8: Assume that Pb > Pe and Py < Pz- Any achievable tuple (D,De) verifies 
the following inequalities: 

D> 1 1 



1+ — 1 + ' 

1 ^ Pb 1 ^ Py 



D e < 1 



1 + — ' 

1 ^ Pe 



De < 



l+S- 

1 1+ p z I 1 _ 1 



- -L 1 1 
T- T — — 



D 1 P E P b 

P Y 

Proof: See Appendix IH1 ■ 
We next propose a hybrid digital/analog scheme based on Theorem |3] that turns out to be 
optimal i.e., that achieves the region of Proposition [8l when Py < Pz and Pb — > oo. 

C. Hybrid Coding 

Proposition [9] below follows from Theorem [3] choosing variables U, V and X as follows: 

U = , (29) 
V = aA + 1 N , (30) 
X = (0A-7N)VP , (31) 



where a G K, /3 G [0, 1), 7 = a/1 — /3 2 and A r ~ A/"(0, 1) is a standard Gaussian random variable 
independent of A. Note that X ~ jV(0, P) writes as a deterministic function of A and V^: 

X = ({a + P)A~V)y/P . (32) 

Function A is defined as the MMSE estimator of A from (V, B, Y). 

The hybrid digital/analog scheme of Section [VI] with variables ([29b— (f3TT) reduces to the one 
depicted in Fig. [T3l 
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{l,...,2 nRf } 



1 



x >^ X n 
P 



(a + f3) 

Figure 13: Hybrid digital/analog scheme for secure transmission of a Gaussian source over a 
Gaussian wiretap channel. 



Proposition 9 (Hybrid scheme): A tuple (D,De) E is achievable if 
1 



D > 



De < t — • min 



Pb 1 Py 



1 + 



Pb 



1 + 7 



P 



for some a E R, (3 E [0, 1) such that 

a 2 P 
7 J + Py 

where 



^ + ^(a + /3) 2 <^ r (l + ^ r 



P 



2 
P B 



7 



(33) 
(34) 



(35) 



(36) 



Proof: See Appendix U ■ 
Remark 3: In the proposed scheme, unlike dirty-paper coding for point-to-point communi- 
cation without secrecy constraint OTTl . the source A (that can be viewed as the state of some 
channel, known at the encoder -see Fig. [9]) and the channel input X are not independent. 



D. Special Case: Py < Pz, Pb — > oo 

From now on, we focus on the unsolved case (represented by "?" in Table IJ). In particular, 
we assume that Py < Pz- Then, if Bob does not have any side information i.e., B = or 
equivalently Pb oo: 

• The hybrid digital/analog scheme of Proposition |9] is optimal and yields Theorem 0] 
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• The digital scheme of Theorem [2] is strictly sub-optimal, as shown by Proposition \W\ and 

Fig.na 

Theorem 4 (Closed-form characterization): If P Y < Pz an d -8 = 0, (D,D E ) e W + 2 is 
achievable if and only if 

1 



D > 



D E < 



1 + -^ ' 

^ Py 



1 



max < 1 ; -k • 



' D l+J- > + 



(37) 
(38) 



i 



Proof: The converse part directly follows from Proposition [8] by letting Pg tend to infinity. 
The direct part follows after Proposition |9] by letting Pb tend to infinity and choosing, for 
any distortion level D e 



i 



/3 



(39) 
(40) 



Details are provided in Appendix HI ■ 
The following proposition provides a simple expression of region 7^ gital i.e., the set of all 

tuples achievable by the digital scheme of Section ITTTl 

Proposition 10 (Digital scheme): If Py < Pz and 5 = 0, (D, D E ) G ^dLitai if an d only if 

' (41) 



D > 



1 + ^ ' 

1 ^ P Y 



De < 



- + — 

M ^ Pe 



mm 



(42) 



for some e 

Proof: The proof of the converse part is given in Appendix [K] 

The direct part follows from Theorem [2] with Gaussian variables U, V, Q and T = X after 
some straightforward derivations. Details are omitted. ■ 

Remark 4: If D > § , then // = 1 is optimal in Proposition ITOl yielding inequalities (1371) , 
(1381) in Theorem @J This implies that the digital scheme of Section [Hi] is optimal in this region. 
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For such distortion levels, the quantity D E 



Var [A\E] is achievable, meaning that Eve 



cannot retrieve additional information from the communication between Alice and Bob. 

In the following, we will also compare the two above schemes with a pure analog one, 
consisting in directly sending a scaled version of the source over the channel. Its performance 
is given by the following proposition. 



Proposition 11 (Analog scheme): If B = 0, (D,De) € 
scheme if 

1 



is achievable through an analog 



D > 



Dp < 



1 + 



Pe 



+ 



(i-i) 



Pz 



Proof: See Appendix O 
Remark 5: If D = D mm 



-, then Proposition QT| yields inequalities (1371) , (1381 ) in 



Theorem |4] i.e., the above analog scheme is optimal. 

Remark 6: When there is no secrecy requirement i.e., D E = 0, all the above schemes can 
achieve distortion D min , as stated in lfT9l . 11321 . 

Numerical Results: Fig. [T4l represents the largest achievable D E as a function of the distortion 
level at Bob D for 

1) the optimal hybrid digital/analog scheme of Theorem H] 

2) the digital scheme of Proposition [10] (optimizing over /x), 

3) the analog scheme of Proposition \TT\ 

for parameter values P = 1, P Y = 0.5, Pz = 1, Pe = 1- 

As a matter of fact, the proposed hybrid digital/analog scheme outperforms both pure analog 

i+-f- 

and digital schemes. Furthermore, while the digital scheme is optimal for D > / (Remark |4|) 

+ Py 

and the analog one for D = D min (Remark [5]), a time-sharing combination of these falls short 
to achieve the entire region, as shown by Fig. [14] and Theorem |H 

Remark 7: While the hybrid digital/analog scheme of Section IIX-CI can be used regardless 
of the values of the parameters, we did not manage to prove a result of optimality in the more 
general case where P B < oo. However, numerical optimization over a and (3 in Proposition |9] 
tends to show that this scheme does not achieve the outer bound of Proposition [8] 
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Optimal (Th. g) 

- Digital (Prop. \M 
Analog (Prop. \TQ) 



0.3 0.4 0.5 0.6 0.7 0. 

D 



Figure 14: Quantity De as a function of the distortion at Bob D (P = 1, Py = 0.5, Pz = 1, 
Pe = 1). 



X. Summary 

In this paper, we have investigated the general problem of source-channel coding for secure 
transmission of sources over noisy channels with side information at the receivers. This setting 
can be seen as a generalization of the problems of secure source coding with side information at 
the decoders, and the wiretap channel. A general outer bound on the corresponding achievable 
region has been derived, as well as two inner bounds based on (z) a pure digital scheme which 
combines secure source coding of [fT3l , IfPfll with coding for broadcast channels with confidential 
messages [3], and (ii) a novel hybrid digital/analog scheme (in the matched-bandwidth case). 

The proposed bounds do not match in general, but the digital scheme turns out to be optimal 
under some less noisy conditions. However, a simple counterexample shows that a joint source- 
channel scheme may achieve better performance in some other cases. At first look, this is not 
surprising since it is well-known that joint source-channel coding/decoding are well-suited for 
broadcast channels without secrecy constraints [|20l . when all decoders must perfectly reconstruct 
the source. But the secure setting is rather different because Alice only wants to help one receiver 
(Bob), while she wants to blur the other one (Eve). Therefore, the intuition indicates that the 
optimal strategy would be the opposite i.e., separation between source and channel encoders, as 
in Propositions CD and [2l On the other hand, the proposed hybrid digital/analog scheme can be 
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useful in terms of secrecy. In a quadratic Gaussian setup when side information is only present 
at the eavesdropper, this strategy turns out to be optimal. However, in a more general case where 
both receivers have side information, a new scheme seems to be needed. 

Appendix A 

Strongly Typical Sequences and Delta-Convention 

Following [|33l . we use in this paper strongly typical sets and the so-called Delta-Convention. 
Some useful facts are recalled here. Let X and Y be random variables on some finite sets X 
and X respectively. We denote by px,y (resp. py\x, an d Px) the joint probability distribution 
of (X, Y) (resp. conditional distribution of Y given X, and marginal distribution of X). 

Definition 4 (Number of occurrences): For any sequence x n E X n and any symbol a E X, 
notation N(a\x n ) stands for the number of occurrences of a in x n . 

Definition 5 (Typical sequence): A sequence x n E X n is called (strongly) 5 -typical w.r.t. X 
(or simply typical if the context is clear) if 

1 

n 

and N(a\x n ) = for each a E X such that px(a) = 0. The set of all such sequences is denoted 
by T?(X). 

Definition 6 (Conditionally typical sequence): Let x n E X n . A sequence y n E y n is called 
(strongly) 5-typical (w.r.t. Y) given x n if 



-N{a\x n ) -p x (a) 



< 5 for each a E X 



-N{a,b\x n ,y n ) - -N{a\x n )p Y \x(b\a) 
n n 



< 5 for each a E X.b Ey 



and, N(a, b\x n , y n ) = for each a E X, b E y such that p Y \x(b\ a ) = 0. The set of all such 
sequences is denoted by T£(Y\x n ). 

Delta-Convention / [33l/ 1: For any sets X, y, there exists a sequence {5 n }neN* sucn tnat tne 
lemmas stated below hold[] From now on, typical sequences are understood with 5 = 5 n . Typical 
sets are still denoted by T 5 n (-). 

Lemma 2 ( K33\ Lemma 1.2.12]): There exists a sequence r] n > such that 

PxCWO) >!-»*»■ 

3 As a matter of fact, S n — > and ^/nS n — > oo as n — > oo. 
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Lemma 3 ( l\33\ Lemma 1.2.13]): There exists a sequence r\ n > such that, for each 

n— >oo 

x n e T«(X), 

1 



n 



log\\T?(X)\\-H(X) 



< Vr, 



-log\\T?(Y\x n )\\-H(Y\X) 
n 



< 



Lemma 4 (Asymptotic equipartition property): There exists a sequence r\ n > such that, 

n— ¥oo 

for each x n G T£(X) and each y n G T£{Y\x n ), 



-\ogp x {x n ) - H{X) 



n 



n 



\ogp Y \ x {y n \x n )-H(Y\X) 



< Vr 



Lemma 5 (Joint typicality lemma / |5?1/ ): There exists a sequence i] n > such that 



-\ogp Y (Tf(Y\x n ))-I(X;Y) 
n 



< r] n for each x n G Tf(X) . 



Proof: 



PY (T?(Y\x n )) = WW) 

y n €T"(Y\x n ) 

< \\T?(Y\x n )\\ 2- n[H{Y) ~ an] 

(b) 

\J 2^[H(Y\X)+I3 n ] 2-n[H(Y)-a n ] 
_ 2 -n[-f(^;^)-^-an] 



where 



step (a) follows from the fact that Tg(Y\x n ) C Tg(Y) and Lemma H for some sequence 



01, 



+ 0, 



• step (b) from Lemma [3l for some sequence f3 n > 0. 

n— >oo 

The reverse inequality p Y (Tg(Y\x n )) > 2~ n [ / ( X;y ) +/3n+Qn l can be proved following similar 
argument. ■ 
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Appendix B 



Graphical Representation of Probability Distributions 



Following ll35l Section II], we use in this paper a technique based on undirected graphs, 
that provides a sufficient condition for establishing Markov chains from a joint distribution. 
Such a technique for establishing conditional independence was introduced in ll36l for Bayesian 
networks, and further generalized to various types of graphs ll37ll . This paragraph recalls the 
main points of this technique. 

Assume that a sequence of random variables X n has joint distribution with the following 
form: 

P(x n ) = fi(x Sl )f 2 {x S2 ) ■ ■ ■ f k {x Sk ) , 

where, for each i 6 {1, . . . , k}, Si is a subset of {1, . . . , n}, notation x Si stands for collection 
(xj)j E Si> an d fi is some nonnegative function. 

A. Drawing the Graph 

Draw an undirected graph where all involved random variables e.g., (Xj)^.^ n }, are nodes. 
For each i 6 {1,. . . ,k}, draw edges between all the nodes in X Si . 

B. Checking Markov Relations 

Let Qi, Q 2 , and Q 3 be three disjoint subsets of {1, . . . , n}. If all paths in the graph from a 
node in Xg 1 to a node in Xg 3 pass through a node in Xg 2 , then Xg 1 -e- Xg 2 -e- Xg 3 form a 
Markov chain. The proof of this result can be found in 0511 and is omitted here. 

Appendix C 
Proof of Theorem □ (Outer Bound) 

Let (k,D,A) be an achievable tuple, and e > 0. There exists an (n,m)-code (F,g) s.t. 




(43) 



E[d(A n ,g(B n ,Y rn ))] < D + e , 
-H{A n \E n Z m ) >A-e, 



(44) 



(45) 



n 



with channel input X m as the output of the encoder F(A n ). 
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From the fact that random variables Aj, Bi, E( are independent across time and the channel 
X ^ (Y, Z) is memoryless, the joint distribution of (A r \ B n , E n , X m , Y m , Z rn ) can be written 
as follows, for each i G {1, . . . , n} and each j G {1, . . . , m}: 

p(a n , b n , e n , x™, y m , z m ) = p(a*-\ V~\ e*" 1 ) p(a h 6 is e*) p« +1 , 6? +1 , e? +1 ) 

Following the technique described in Appendix [B] and using the above expression, we can obtain 
the graph of Fig. \\5\ that will be used to establish Markov chains. 

For each % G {1, . . . , n} (resp. each j G {1, . . . , m}), define the source (resp. channel) auxiliary 
random variables [/j, Vi (resp. Qj, Tj) as 

Ui = ( B^E l -\Z m ) , (46) 

Vi = (A 1 ' 1 , B l -\ B? +1 , E { ~\ Y m ) , (47) 

Qi = { B n ,Yi-\Z™ +1 ), (48) 
Tj — (A n ,B n ,Y j ~ 1 ,ZJ\. 1 ) . (49) 

Note that (U i: Vi) -e-Aj-e- (Bi, E t ), and Qj -e- 2} -e- X,- -e- (Y,-, Z,-) form Markov chains (see 
Fig. US). 

Following the usual technique, we introduce independent random variables K and J, uniformly 
distributed over the sets {1, . . . , n} and {1, . . . , m}, respectively. We also define random variables 

A = A K , B = B K , E = E K ,U = (K,U K ), V = (K,V K ), X = X Jt Y = Y Jt Z = Zj, 
Q = ( J, Q 3 ) f and T = (J, Tj). (U, V)-e- A-e- (B, E) and Q -e-T -e- X -e- (Y, Z) still form 
Markov chains. (A, B, E), resp. (X, Y, Z), is distributed according to p(abe), resp. p(xyz), i.e., 
the original distribution of (A4, B iy Ej), resp. (Xj, Yj, Zj). 
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Figure 15: Outer bound-Graphical representation of distribution p(a n b n e n x m y m z m ). 



A. Rate 

We first prove the rate inequality © in Theorem Q] 

n 

("I 

i=l 
n 

(&) 



/(A n ; y m |s n ) l = J ^ /(A; Y m \A i ~ 1 B n ) 
i=i 

n 

^ J(A; Y m \A i ~ 1 B n E i ~ 1 ) 
1=1 

i=i 

( J A^ 1 B^ 1 Bt +l E i ~ 1 Y m \B i ) 



i=i 



where 



step (a) follows from the chain rule for conditional mutual information, 
step (b) from the Markov chain (A, ^ m ) -e- (A* -1 , 5 n ) -e- E^ 1 (see Fig. Q3), 
step (c) from the independence of random variables A+, Bi, and Ei across time, 
step (d) from definition (|47l) . 
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We now find an upper bound for the latter quantity: 



I(A n ;Y rn \B n ) ( = } I{A n ; Yj\B n Y j ~ 1 ) 



3=1 



(6) 



< i{ An B nYj ~ lz T+^ Y l 



3=1 

m 



< 52l{T j; Yj) , 

3=1 



where 



• step (a) follows from the chain rule for conditional mutual information, 

• step (6) from the non-negativity of mutual information, 

• step (c) from definition (|49l ). 

n m 

Putting all pieces together, we proved that ' s ^ j I{Ai]Vj\B i ) < I(Tj] Yj 

i=l j=l 

variables K and J, this inequality writes 

n m 

HAk\ Vk\B k , K = i)<J2 I{Tj\ Yj\J = j) , 

i=l j=l 



i.e., 



Tfl 

I(A;V\B)<-I(T;Y) . 
n 



B. Distortion at Bob 

Bob reconstructs g(B n ,Y m ). The 2-th coordinate of this estimate is 

g^^B^^^B^^MV^B,) . 

The component-wise mean distortion at Bob thus writes: 

E [d(A n ,g(B n , Y m ))] = - E ^04 ^)) 



i=l 



Using random 



(50) 



K = i 



= -J2E\d(A K ,A K (VK,B K )) 

i=l 

= E[d(A,i(V,5))] , 
where we defined function A on V x B by A(V, 5) = A(K, V K , B K ) = A K (V K , B K ). 



(51) 
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C. Equivocation Rate at Eve 

We expand the equivocation at Eve H(A n \E n Z m ) in two ways. 
1) : First, 

n 

(a) 



H(A n \E n Z m ) ^^H{Ai\A^ x E n Z m ) 



(*>) 



(c) 



1=1 

n 



H{A i \A , l +l B^ +l E i Z 1 



n 



< J2h(A.\b? +1 e*z* 

i=l 
n 

^H(A i \U i E i ) , 



(J 
i=i 



where 

• step (a) follows from the chain rule for conditional entropy, 

. step (b) from the Markov chain A { (Af +1 , E\ Z m ) -e- (B? +l , £f +1 ) (see Fig. [B]), 

• step (c) from the fact that conditioning reduces the entropy, 

• step (d) from definition (|46l) . 

Using random variable K defined above, the equivocation rate at Eve can be bounded as follows: 

1 n 

-H(A n \E n Z m ) < - y^H(A K \U K E K ,K = i) 
n n / — ' 

i=l 

= H(A\UE) . (52) 
2) : Second, from standard properties of conditional entropy and mutual information: 

H{A n \E n Z m ) = H(A n \B n Y m ) + I{A n ; B n Y m ) - I{A n ; E n Z m ) 
= H(A n \B n Y m ) + I(A n - Y m \B n ) + I(A n ; B n ) 

-I(A n -E n \Z m )-I(A n -Z m ) 
= H(A n \B n Y m ) + I{A n ; Y m \B n ) + I{A n Z m ; B n ) 

- I(A n - E n \Z m ) - I(B n A n - Z m ) 
= H(A n \B n Y m ) + I(A n - B n \Z m ) - I(A n - E n \Z m ) 



+ I(A n ; Y m \B n ) - I{A n ; Z m \B n ) , (53) 
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where step (a) follows from the Markov chain B n -e- A n -e- Z m . 

We now separately study the "source" term A s and the "channel" term A c . 

a) Study of A s : The "source" term A s can be studied following the argument for the 
converse part of [Q~4, Theorem 3] (see [fT4l Appendix E-C]): 

A s = H(A n \B n Y m ) + I{A n ; B n \Z m ) - I{A n ; E n \Z m ) 

( = } H(A n \B n Y m ) + I(A n - B n ) - I(Z m - B n ) - I(A n - E n ) + I(Z m - E n ) 

n 

{ => [mMX^Y™) + I{A % - B t ) - I(B? +1 Z m ; B t ) - J(A; E,) + I{E i ~ x Z m ] E t ) 
i=i 

n 



[H(A i \A i - 1 B n E i - 1 Y m ) + I{A % - B t ) - I(B? +1 Z m ; B { ) - /(A; E, 



i=\ 



+ I{E i - 1 Z m ] Ei) + I(Ei, B^ 1 \E i - x Z m ) - I(B t - E^B^Z 171 ) 



i=l 
n 

(J 

/ - I 

where 



= [H(A l \A i - 1 B n E t - 1 Y m ) + I(A f , B<) - I(A t ; E t ) 

i=l 

+ I(E t ; B r l +l E % - l Z m ) - I(B, t ; B^E^Z™ 

n 

( = } [HiAilViBi) + I(A t ; Bt) - I(A t ; E t ) + I(E f , U t ) - I(B t ; U, 
i=i 

n 

[HiAilUiEA - (l{Ai- Vi\B t ) - I(A; U^B,)) 



(54) 



step (a) follows from the Markov chain (B n , E n ) -e- A n -e- Z m , 

step (6) from the chain rules for conditional entropy and mutual information, and the fact 
that random variables A*, Bi and Ei are independent across time, 

step (c) from the Markov chain A, -e- B n , F m ) -e- E l ~ x (see Fig. [15]) and Csiszar 

and Korner's equality [|3l Lemma 7], 

step (d) from definitions (1461) and (1471) . 

step (e) from the Markov chain Ui -e- -e- (Bi, Ei). 
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b) Study of A c : The "channel" term A c can be studied following the argument for the 
converse part of 01 Theorem 1] (see [3, Section V], [7, Section 2.4]): 

A c = I(A n ; Y m \B n ) - I(A n ; Z m \B n ) 

m 

( = ] ^ [l(A n ; YjlB^- 1 ) - I(A n - Zj\B n ZY +1 ) 

3=1 
m 

= [l{A n Z^ 1 ;Y j \B n Y j - 1 )-I(Z^ 1 ]Y j \A n B n Y j - 1 ) 



3=1 



- I{A n Y j ~ l ] Zj\B n Zf +1 ) + I{Y j - X ] Z j \A n B n Z r - 

m 

= IWZj+il Yj\B n Y j - 1 ) - I(A n Y j - 1 - Z,j\B n Zf +1 



m \ 
j+U 



3=1 
m 



Y [l(A n ; Y 3 \E n Y^ x Z^ x ) + I(Z? +1 ; Yj\B n Y j ~ l ] 

3=1 



- I(A n ; Z j \B n Y j - 1 Zf +1 ) - I{Y j - 1 ; ZAB n Z r - 

m 

~ Yl l 1 ^' Yj\B n Yi- l Z™ +l ) - I(A n ; Z^Y^Z™^ 



m ^ 
3+1 J 



(d) 



3=1 
m 



Y^rrVjQ,) ■ liTrZ, (),) 
3=1 



where 



(55) 



• step (a) follows from the chain rule for conditional mutual information, 

• steps (b) and (c) from Csiszar and Korner's equality [31 Lemma 7], 
. step (d) from definitions ([48]) and (|49l) . 

c) : Gathering (I53l)-(l55l). using variables J, K, and new source-channel variables, the 
equivocation rate at Eve writes: 



-H{A n \E n Z m ) = H(A\UE) - I (A; V\B) - I (A; U\B 



n 



-^(l{T-Y\Q)-I{T-Z\Q)) 



■ (56) 



D. End of Proof 

Inequalities (I50l)-(l52l). and (l56l) only involve marginal distributions of auxiliary variables 
p(uv\a) and p(qtx). Consequently, we can define new variables U, V, Q, T, X, with identical 
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marginal distributions p(uv\a) and p(qtx) (and hence verifying (150l)-(l52l), and (1561) ) such that 
the (global) joint distribution writes piuvqtabexyz) = p(uv\a)p(abe) p(q\t)p(t)p(x\t)p(yz\x) i.e., 
such that source and channel variables are independent. 

Gathering inequalities (|50l) -(l52l). (|56l) . and (|431)-(|45T). we then proved that, for each achievable 
tuple (k, D, A) and each e > 0, there exist random variables U, V, Q, T, X with joint distribution 
piuvqtabexyz) = p(uv\a)p(abe)p(q\t)p(t)p(x\t)p(yz\x), and a function A such that 

I{A;V\B)<(k + e)I(T;Y) , 
D + e > E[d(A, A(V, B))] , 

A - e < H{A\UE) - A\B) - I{U; A\B) - (k + e) (l{T; Y\Q) - I(T; Z|Q))] , 

i.e., (k + e, D + e, A — e) e 7^ out . Letting e tend to zero then proves Theorem [Q ■ 

Appendix D 

Proof of the Converse Part of Proposition ED 

Let (A; = 1, -D = 0, A) e 72-digitai i- e - 5 suc h that there exist random variables U, V, Q, T, X 
with joint distribution p(uvqtaexy z) = p(u\v)p{v\a)p{ae)p(q\t)p(tx)p{yz\x), and a function A, 
verifying 

I(U;A\B)<I(Q;Y) , 
I(V;A\B)<I(T;Y) , 
> E[d(A, A(V, B))] , 

A < H(A\UE) - [l{V; A\UB) - (l(T; Y\Q) - I{T; Z|Q))] + . 
From the assumptions of Section |Vj we can easily prove the following inequalities: 

P(1-H(A\U)) <1-H(X\Q) , (57) 
A<H(A\U)+h 2 {e)-H(E\U) 

PH(A\U) - (H(X\Q) - H(Z\Q) + h 2 (C) 



(58) 

Since < H(A\U) < H(A) = 1 and < H{X\Q) < H{X) < 1, we now introduce two 
parameters: u = h 2 x (H(A\U)), q = h 2 l {H(X\Q)). 
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Then, from the fact that E is the output of a BSC with crossover probability e and input A, 
Mrs. Gerber's lemma ll38l yields: 

H{E\U) > h 2 (e*u) . (59) 
Similarly, since Z is the output of a BSC with crossover probability ( and input X: 

H(Z\Q)>h 2 ((*q) . (60) 

Gathering ([57])-(l6Q]), we obtain 

(3{l-h 2 {u)) <l-h 2 {q) , 

A < h 2 (u) + h 2 (e) -h 2 (e*u)- 0h 2 (u) - (h 2 (q) -h 2 ((*q) + h 2 (() 
This proves the converse part of Proposition HI ■ 

Appendix E 
Proof of Lemma CD 

In this section, we prove Lemma Q] following the argument of [|24|. In the decoding procedure 
described in Section IVII-Cl an error occurs in the first step if there exists another admissible 
codeword w n (r^) with r[ r x . The probability of this event writes 



P dA 4 Pr {3 r'^n: (u n (r[), B\ Y n ) e Tf(U, B, Y)} 



2 n l 



r[=l 



2™"1 



r[=l a" 



We now study each term of the above summation: For each r[, and each a n , 



Pr [n ? r[, (u n (r[), B n , Y n ) G T?(U, B, Y) 
<Pr{(u n (r[),B n ,Y n )eTnU,B,Y) 



A n = a n 
A n = a 



x Pr {5™ = b n ,Y n = y n A n = a n ,r 1 ^r' 1 } . 



(61) 



Pr{u n (r' 1 )=u n ,B n = b n ,Y n = y n A n = a n ,n ^ r[} 

(u n ,b n ,y n )£T"(U,B,Y) 

{u n {r[) = u n A n = a n , ri ^r[} 

(u«,b n ,y«)£T™{U,B,Y) 



(62) 
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For each r' x , and each a n , according to the encoding procedure described in Section IVII-B1 

Pr {n = r[ | ^ = a"}=Pr j ( P| {u n (k) (£ Tg(U\a n )} \ D {u n (r[) G T£(U\a n )} 



k=l 



< Pr {u n (r[) G T£(U\a n )} 

< 2-n(I(U;A)~'ri„) 

where the last inequality holds for some sequence i] n > (see Lemma [5] in Appendix lAl) 

n— >oo 

from the fact that the codewords are uniformly distributed over Tg(U), independent of the source, 
and a" G Tg(A). From this inequality, there exists k < 1 such that, for some sufficiently large 

n, 



A n = a n < k . 



Pr |ri = 

The above upper bound yields the following inequality, for each u n , r[, and a n : 

Pr {u n (r[) = u n A n = a n ,n ^ r^j 

= Pr {u n (r[) = u n 
Pr {u n (r[) = u n } 



(63) 



A n = a r 



Pr < 


n ^ r[ 


A n = a n ,u n {r[) = m"} 


1 -Pr-j 


ri = r[ 


A n = a n | 



< 



1 - K 



(64) 



where the above equation follows from (1631) and the fact that u n (r[) and A" are independent. 
Plugging (|64l ) into (|62l) yields, for each r l5 and each a n , 



Pr {n / r;, (u"(r[), B", Y") e 77(17, B, Y) 



(u»,))°,f)eT;([7,B,y) 



A n = a n } 
Pr-jV^) = u n } 

l-K 



x Pr {B n = b n ,Y n = y n A n = a n , n ^ r[} . (65) 

From (|63l) once again, the last term in the r.h.s. of the above equation can be upper bounded 
as follows, for each r[, each a n , and each (b n ,y n ) G Tg(B,Y): 



Pr{B n = b n ,Y n = y n 



A n = a n ,n 



Pr < 




A n = a n | 


Pr < 


n _^ r ^ 


A n = a n 





< 



Pr jfi n = 6 n , Y n = y n A n = a n | 



1 - K 



(66) 
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Gathering (16TT ), (1651 ) and (|66l ), we obtain, from the fact that the codewords are identically 
distributed, 

^ ' (u n ,b n ,y n )£T?(U,B,Y) 

Finally, from the joint typicality lemma (Lemma \5\ in Appendix [A]), there exists a sequence 
r]' n > such that 

n->oo 

P, , < Z 9 -n(/(f/;By)-t/J 
(1 — K) z 

This proves Lemma \T\ ■ 

Appendix F 

Proof of the Converse Part of Proposition [6] 

Assume that P B < P E , and let (D, D E ) be an achievable tuple. From Proposition \T\ there 
exist random variables V, Q, T, X with joint distribution p(vqtabexyz) = p(v\a)p(abe) p{q\t)p(t) 
p(x\t)p(yz\x), and a function A, verifying 

I(V;A\B)<I(T;Y) , 
D > E[d(A, A(V, B))] , 

A < h{A\E) - A\B) - J(T; + J(T; Z|Q) 

Var [X] < P , 

where A = | log(27re D E ). 

Then, from standard properties of differential entropy, and from the fact that distortion mea- 
sure d is the Euclidean distance on R, the following sequence of inequalities holds true: 

2 2h{AlVB) /(2ire) < \ar[A\VB] < E[d(A, A(V, B))] , 

and hence 



2 2h{A\VB) - 1 + 1_ D 
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Gathering the above equations, tuple (D,D E ) verifies the following inequalities: 

1 1 P 

D E < ' 



1 + — ' 

^ Pe 



1 + — 

D < Z P B . D . 2 21UT;Y\Q)-I(T;Z\Q)] 

1 + — 

1 ^ Pe 

Depending on the values of P Y and P z , there are two different cases, investigated in the 
following paragraphs. 

A. P Y < Pz 

Assume in this section that Py < Pz- According to Remark [2l the Markov chain X -e-Y -e- Z 
can be assumed without loss of generality. Hence, from the long Markov chain Q -e- T -e- X — 
<^Y -e- Z, I(T;Y\Q) - I(T;Z\Q) = I(T,Y\QZ) < I(X;Y\Z), and tuple (D,D E ) verifies 
the following inequalities: 

1 1 P 
1 + — 

1 ^ Pe 

Then, from the Markov chain X-e-Y-e-Z, there exists a random variable N z ~ A/"(0, Pz—Py), 
independent of Y, such that Z = Y + Nz- The entropy power inequality (EPI) 031, P4ll yields: 

22h(Z) y 2 2h ( Y ) _|_ 2 2/l (^z) 
From the latter equation and inequality Var [Y] < P + Py, 

I(X; Y\Z) = h(Y) - h(Y\X) - h(Z) + h(Z\X) 

< h(Y) - h{N Y ) - l - log (2 2h ^ + 2 2h ^ + h(N z ) 
1, / 1 P z 
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Tuple (D,D E ) thus verifies the following inequalities: 

1 1 P 

DE< -rh' 

D E < ^ • D Py 



B. Py > P Z 

Assume in this section that P Y > P Z - Then, I(T;Y\Q) < I(T;Z\Q), and tuple (D,D E ) 
verifies the following inequalities: 

1 1 P 
- < 1 + — 



1 + ^ D~ Py' 
L ^ Pe 
1 ^ Pe 

This concludes the proof of the converse part of Proposition |6j 

Appendix G 

Proof of the Converse Part of Proposition [7] 

Assume that P E > Pe and Py > Pz, and let (D,De) be an achievable tuple. From 
Proposition |2l there exist random variables U, V, X with joint distribution p(uvabexyz) = 

p(u\v )p(v \a)p(ae)p(b\e) p(x)p(yz\x)\\ and a function A, verifying 

I{V-A\B)<I{X-Y) , 
D > E[d(A,A(V,B))] , 
A < h{A\VB) + I(A; B\U) - I (A; E\U) , 
Var [X] < P , 
where A = \ log(2?re D E ). 

4 Since it is assumed that Pb > Pe, according to Remark[2] the Markov chain A -e- E -e- _B can be assumed here without 
loss of generality. 
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From the long Markov chain U^V^A^E^B, I (A; B\U)-I(A; E\U) = -I (A; E\UB) < 
—I(A]E\VB), and tuple (D,A) verifies the following inequalities: 

I(V;A\B)<I(X;Y) , (68) 

D > E[d(A, A(V, B))] , (69) 

A < h(A\VB) - h(E\VB) + h(E\AB) , (70) 

Var[X]<P. (71) 

Moreover, the side informations write: 

E = A + N E , 

B = A + N E + N B = E + N B , 

where N B ~ Af(0, P B - P E ) is independent of A and N E ~ Af(0, P E ). 

In order to find an upper bound on the r.h.s. of (170)) . we need the following expansion of E, 
for any 7 6 1: 

E = 7 5 + (1 - i)A + C , (72) 

where 

C=(l- j)N E - 1 N B . 

Note that (A, B, C) is a Gaussian vector, and that A and C are independent for any 7. The 
usefulness of the above expansion comes from the fact that C is also independent of B if 

7 = Ek. 
• p b - 

E[BC] = (1 — 7) E [BN E ] - 7E [BN B ] 
= {i-j)P E - 1 (p B -P JS ) 

= P E - jP B 
= . 

Finally, since V only depends on A, C is independent of (V,A,B). 
Using expansion (1721 , we now write 

h(E\VB) = h{^B + (1 - 7 )4 + C|VS) 
= - 7 )A + C|ra) . 
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And from the above paragraph, the conditional EPI holds between A and C (given (V,B)): 

22h((l-f)A+C\VB) > 22fc((l-7)A|V-B) _|_ 2 2h ( c \ VB ) 

Since C is independent of (V, A, B), the last entropy writes 

h{C\VB) = h{C\AB) 
= h(E\AB) , 

where the last equality follows from expansion (l72l) . 
Gathering the above equations, (1701) yields 

A < h(A\VB) - l - log ((1 - 7 ) 2 2 2h ^ VB ^ + 2 2h ^ AB ^) + h(E\AB) 
= ^ log i 

= l 1 °s\ ^- ^ e i i 1 ' (73) 

* \ 2 2h(A\VB) I - p B p s 

where the last equality follows from Var [i?|AB] = ^1 — p| j after some manipulations. 

Now, from standard properties of differential entropy, and from the fact that distortion mea- 
sure d is the Euclidean distance on R, the following sequence of inequalities holds true: 

2 2/l(A|yB) /(2vre) < Was[A\VB] < E[d(A, A(V, B))] , 

and hence 

2 2h(A\VB) ~ 1 + A_ D ■ 

Gathering the above equations, tuple (D,De) verifies the following inequalities: 

1 1 P 

IT^J ' r>- 1 + Py ' 

1 

Dk < 



1 + J L 

D ^ P E P B 



This proves the converse part of Proposition |7J 
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Appendix H 
Proof of Proposition [8] 

Assume that Pb > Pe and Py < Pz, and let (D,De) be an achievable tuple. From 
Theorem [H there exist random variables U, V, Q, T, X with joint distribution p(uvqtabexyz) = 
p(uv\a)p(abe) p(q\t)p(tx)p(y\x)p(z\y)\\ and a function A, verifying 

I(V;A\B)<I(T;Y) , 

D > E[d(A, A(V, B))] , 

A < h(A\UE) - [l(V; A\B) - I(U; A\B) - (/(T; Y\Q) - I(T; Z|Q))] + , 

Var [X] < P , 

where A = \ log(2vreL> s ). 

From the Markov chain Q -e- T -e- X -e- Y -e- Z, tuple (£), A) verifies the following 
inequalities: 

</(X;Y) , (74) 

D > E[d(A,i(V,B))] , (75) 

A < h(A\UE) — \l(V; A\B) — I(U;A\B) — I(X; Y\Z)] , (76) 

Var [X] < P . (77) 

Moreover, from the proof of Theorem Q] (see (1461) . (1471) in Appendix O, we can restrict our 
attention to auxiliary variables U, V s.t. U -e- V" -e- A -e- E form a Markov chain. 

We introduce two parameters: v = 2 2h{A ^ VB) /(2ne), fi = 2 2h< ^ A \ UB) / (2-ne). From the fact that 
conditioning reduces the entropy and classical properties of the differential entropy, the above 
parameters are bounded as follows: 

u - ^ - T^X • (78) 

l+ Pb 

We now write (1741)— (1771) as functions of these parameters. First, recalling that distortion 
measure d is the Euclidean distance on M, 

E[d{A, A(V,B))] > Var[A\VB] > v . (79) 

5 Since it is assumed that Py < Pz, according to Remark [2] the Markov chain X -e- Y -e- Z can be assumed here without 
loss of generality. 
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Since Pb > Pe, an d according to Remark [2l we can assume that A-e-E-e-B form a Markov 
chain. Then, following the argument of Appendix [G] (based on expansion (|72l) together with the 
conditional EPI), we can easily prove the following equation (similar to (17310: 

h(A\UE) < h(A\UB) - - log ((1 - 7 ) 2 2 2h ^ UB) + 2 2h ^ AB) ) + h(E\AB) 

Z V a* + Pe Pb J 

Since the Markov chain X -e- F -e- Z is assumed, (l67l) also holds here: 

/(X;y|Z)<ilogf^5j . (81) 

Gathering the above equations, tuple (D,D E ) verifies the following inequalities: 

1 



v < \x < 



^ Pb 



1 1 P 

D > v , 



j_ _ j_ i_ ' 

Pe Pb 



D e < 



v 1 + 



p 

Py 



t 1 Pe Pb ^ 1 + Pz 

Eliminating parameter v and ji proves Proposition [8] 



Appendix I 
Proof of Proposition [9] 

In this section, we prove a sequence of lemmas which together prove Proposition^ To that end, 
using auxiliary variables (f29T)— (f3TT) . we show that any tuple (D,D E ) verifying conditions (1331- 
(1351) in Proposition |9] lies in region TZL hrid . 

A. Conditional Covariance of Gaussian Variables 

The following lemma can be found in [39, Appendix A. 2]: 
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Lemma 6 (Conditional covariance matrix of Gaussian vectors): Let P, Q be two jointly Gaus- 
sian random vectors with covariance matrix 

A C T 
C B 

Then the conditional covariance matrix Vm\Q\ of P given Q verifies the following equality 



PQ 



(P\Q) 



A - CB~ l C 



l/TT 



(82) 



From the above lemma, we can easily derive the following corollary, which gives the condi 
tional variance for two scalar Gaussian random variables. 

Corollary 1 (Conditional variance of Gaussian variables): Let P and Q be two jointly Gaus 
sian random variables. Then 

Var[P|Q] 



det Tp Q 



Var[Q] 



(83) 



B. Preliminary Lemmas 

Lemma 7: With definition (1301) . 



a 



nv-,A) = -log + - 

Proof: From definition (1301 ), the covariance matrix of (A, V) is given by 

1 



(84) 



AV 



a 

2 i 2 

a a + 7 



Lemma [7] then directly follows from equality 

I(V; A)= l - log 

and Corollary Q] 

Lemma 8: With definitions (|3~0i ([31) . 



Var 



Var[A|V]7 ' 



Var | BY] = 7 2 ' /; " 1 



Proof: From definitions (1301 ), 0Tb . the covariance matrix of (V, S, K) is given by 

a (a/3-7 2 )VP 
i ,<n = a 1 • />/; 

(a/3-7 2 )^P P + Py 



(85) 



2 i 2 

a +7 
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This equation comes from the following sequence of equalities, using (1321 ): 

E[VY) = E[VX) 

= ((a + P)E[VA] - Var [V] ) VP 
= (a/3 - 7 2 ) VP . 

Lemma [8] then follows from Lemma [6] after some straightforward manipulations. ■ 
Letting P B tend to zero in the above lemma yields the following corollary (which can also 

been proved independently using similar argument): 
Corollary 2: With definitions (l30|) . (IITT) . 

Ndx[V\AY] 
Lemma 9: With definitions (f30l) . < f3Tb . 

VarfAlEy] 



(86) 



Proof: From definitions (|30l ), (1311 ), the covariance matrix of (A, B, Y) is given by 

i i pVP 



(87) 



ABY 



1 1 + P B /3\/P 

pVP pVP p + p Y 

Lemma [9] then follows from Lemma [6] after some straightforward manipulations. 
Lemma 10: With definitions (f30|> . (1311) . 



7(X;Z|S) = -log 



(88) 



Proof: Lemma ITOl directly follows from Corollary [6] together with equality Var [Y\X] = P Y , 
expansion 

I(X; Z\E) = h(Z\E) - h(Z\X) , 

which comes from the Markov chain Z-e-X-e-E, and the following expression of the covariance 
matrix of (Z,E): 

' P + P z f3VP 

ZE — , — 

pVP I + Pe 
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C. End of Proof 

We now combine the above lemmas to prove that the inequalities <[T9l)-(l22l) and (|28l ) are 
verified by variables (|29|)-(|3D) under conditions (|33l) - (|35l) . 

As a matter of fact, inequality (fT9l) is verified with definition (|29l ). From (I3TT) and (|36l) . 
X ~ Af(0, P) and the power constraint (|28T) is also verified. 

ij Proof of d20l> : From Lemma [8] and equality Var [V] = a 2 + 7 2 , /(V; SV) writes 

1. //. o?\ l + A + ^ + ft 



This equality together with Lemma [7] and constraint (1351) proves (1201) . 

2j Proof of (|2TI) .- In the quadratic Gaussian case considered in Section [IXl distortion measure 
d is the Euclidean distance on R: 

E[d(A,A(V,B,Y))] = e\(A- A(V,B,Y)f 

Moreover, in the proposed scheme, function A is the MMSE estimator of A from (V,B,Y), 
therefore 

E[d{A,A(V,B,Y))] = Var [A\V BY] . 
We now use the Markov chain V -e- (A, Y)-e-B to expand the following conditional entropy: 
h(A\VBY) = h(A\BY) + h(V\AY) - h(V\BY) , 
and since the above random variables are jointly Gaussian, this yields 

Gathering Lemmas [H |9] and Corollary [2l the above equation writes 

1 



E[d(A,A(V,B,Y))] 



and hence (|2TI) is verified under constraint (1331). 
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3) Proof of (|22l) : From Corollary \T\ with the covariance matrix T AE given below, Lemmas [7] 
and [Kjl we can easily prove the following equality: 



h(A\E) - I(V; A) - I(X; Z\E) = - log 




2vre 



1 + — + — ( 1 + T 
1 ^ p e ^ p z 1 1 ^ 




(90) 



AE 



1 1 

1 1 + Pe 

Then, letting P B tend to zero and replacing P Y by Pz in (1891) yields the following equality 
(which can also been proved independently using argument similar to the one Lemma [8]): 

,2 



I(V;AZ) = -log 



1 . f ( a z \ ( o P 
1 + — 



1+7 



(91) 



Inequality d22j) then follows under constraint (O from (f89~l)-(l9lT) and definition P s = 2 2A /(27re) 
This concludes the proof of Proposition |9j ■ 



Appendix J 
Proof of the Direct Part of Theorem 0] 

Letting P B tend to infinity, (|33l) - ((35l) write 

1 



D > 



l + $ + £(« + /3) 2 



D E < 



1 + 



1 + ^: + 



Pe 1 Pz 



1 + 



■ mm 



Py 



l + $ + £(a + P) 



1 + 7 



P? 



a P , m2 p 



P, 



Y 



(92) 
(93) 
(94) 



We then check that these equations are verified with definitions (1391 ), (1401) under constraint (1381) . 



Recall that we consider here any distortion level D G 



i 
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A. Proof of (El 

From definitions ( |39l ), (|40l ), on one hand: 



a 



( - 2 1 ( P p \ o n ,2 p \ 



l\\l5[-k-T- z )-^PT 



Py 



1 + 7 



2_P_ 



7 



(l + 7 2 ^ 



Py y 



on the other hand: 



(l+7 



2_P_ 

Py 



The denominator in (1921 ) thus writes 

,2 p 



ft- 



7 *V 



1 + 



£> I Py Pz 7 ^ 7 ' \Py) ^ P Py D Py V Py 



P 

Pz 



ii( P_ - pS\ _l «2 _P_ 

£> \ Py Pz)^ P Py 



l + — + 

1 ~ Py ^ D Py P> P Z 



(95) 



where the last equality follows from (1361) . 
Now, from definitions <|36]), (l40l) : 



1 + 7 



(96) 



and hence (1951) writes 



cv 



P 



7 2 ^ 



J_ , Y_p_ 

D P Py 



i 

D 



(97) 



This proves (1921 . 
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B. Proof of (El 

First, from (l96l ) and <|97T ), the two arguments of the min{- ; ■} in (l93l) are equal: 



1 + * 



1 + $ + + ft 2 ^ ' p - 



Y 



9 P 

= !+7 2 p- 

Then, from (l96l) once again, the first term in the r.h.s. of (l93l) writes 

1 1 



J 



and since (1381) writes, for D < 



i+— ' 



^< rr-p > 



P- 

this proves (1931) . 



C. Proof of ^ 

Inequality d94]) directly follows from ([97]) and D > 

This concludes the proof of the direct part of Theorem |4j ■ 

Appendix K 

Proof of the Converse Part of Proposition \10\ 

Let (D,D E ) G 7^ gital i.e., such that there exist random variables U, V, Q, T, X, and a 
function A, with joint distribution p(uvqtaexyz) = p(u\v)p(v\a)p(ae)p{q\t)p(tx)p(y\x)p(z\y))t 
and verifying 

I{U-A)<I{Q-Y) , 
I(V;A)<I(T;Y) , 
D > E[d(A, A(V))] , 

A < h{A\UE) - [l(V; A\U) - (l(T; Y\Q) - J(T; Z\Q)J\ , 
Var [X] < P , 

6 Since it is assumed that Py < Pz, according to Remark[2] the Markov chain X -e- Y -e- Z can be assumed here without 
loss of generality. 
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where A = | log(27re D E ). 

From the Markov chain Q -e- T -e- X -e- Y -e- Z, tuple (£>, A) verifies the following 
inequalities: 

I(U;A)<I(Q;Y) , (98) 

J(V;A) </(X;F) , (99) 

D > E[d(A, A(V))] , (100) 

A<h(A\UE)-\l(V;A\U)-I(X;Y\QZ)] , (101) 

Var[X]<P. (102) 

We now introduce three parameters: v = 2 2h ( A ^ /(2ne), fi = 2 2hi ~ A ^ /(2vre), C = 
2 2/i(r|Q) /(27re). Since J7 -e- -e- A -e- £ and Q ^ fonn Markov chains, from the fact 

that conditioning reduces the entropy and inequality Var [Y] < P + P Y , the above parameters 
are bounded as follows: 

v < \i < 1 , (103) 
Py<C<P + Py- (104) 

We now write (|98T)- (I102I) as functions of these parameters. First, recalling that distortion 
measure d is the Euclidean distance on M, 

E[d(A, A(V))] > Vai[A\V] > v . (105) 

Then, from the Markov chain U -e- A -e- £7, we write 

= /i(A|E7) - /i(£|E7) + h(E\A) . (106) 

Now, since E = A + N E with N E independent of A (and U), the conditional EPI 11341 yields: 

2 2h(E\U) > 2^{A\u) + 2 2/i(JVj5) _ (XQ7) 

Gathering (|106l) and (11071) . we obtain: 

/i(A|f/E) < h(A\U) - i log (2 2h{A ^ + 2 2h ^) + 

-^(^)- 
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From the Markov chain Q-e-X -e-Y -e-Z, there exists a random variable N z ~ A/"(0, P^ — Py), 
independent of (Q, X, Y) such that Z = Y + N z . Then the conditional EPI 01 yields: 

2 27i(Z|Q) > 2 2h (Y\Q) _|_ 2 2h (tiz) 

From the latter equation, 

/(X; F|QZ) = /i(K|Q) - h(Y\X) - h{Z\Q) + /i(Z|X) 

< h{Y\Q) - h(N Y ) - - log (2 2h ( y l«) + 2 2/l ^) + 

< - lor 



2 °VC + ^-iViV, 

Gathering the above equations, tuple (D,De) verifies (11031) . (11041) . and 

1 g + Py 

M- C ' 

1 P 
- < 1 + — , 
1/ ~ Py 

D >v , 



I + J_ ' 

Eliminating parameters (, [i and removing redundant inequalities in the above system prove the 
converse part of Proposition [TOl ■ 

Appendix L 
Proof of Proposition dH 



1 



The analog scheme of Proposition QT| then 



Consider any distortion level D 6 

/ ' TV 

consists in sending a scaled version of the source over the channel: 

X = , (109) 

where r = Py (~ — l). Note that, since P > 1 P , Var [X] = t < P and the power 
constraint (1281) is verified. Bob then simply computes the MMSE estimate A of A from F. 
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In such an analog scheme, the mean distortion at Bob writes 

E[d{A, A(Y))] = Var[A\Y] 

Py 
r + Py 

= D, 

where the next-to-last equation follows after Corollary Q] together with the covariance matrix of 

(AY): 

1 V T 
r + P Y 

The equivocation rate at Eve is h(A\EZ) and quantity D E then writes 

D E = Yar[A\EZ] 
1 



AY 



1 + — + — 

1 ^ P E ^ P Z 



1 + — + (~ ~ I) ^ ' 

1 ^ Pe ^ \D L ) P z 

where the next-to-last equation follows after some straightforward manipulations from Lemma [6] 
and the covariance matrix of (A,E, Z): 

i i 

\t:z = 1 1+ Pe Vt 
St v 7 ^ t + P y 

This proves Proposition [TT] 
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